With the explosive growth and popularity of the Internet, more and more enterprises are looking towards building their network infrastructure across the Internet without having to spend a lot on private leased lines. However, with the increasing usage of the Internet to build networks, more and more "evil ways" of breaking into the network to gather sensitive information are also evolving. Therefore, security on the Internet has been a main concern for each enterprise. Enter IP Security or IPSec. IPSec provides the necessary infrastructure to extend an enterprise's private network across the Internet to reach out to customers and business partners, in other words, to build what is called a "virtual private network (VPN)".
IPSec is a set of protocols and standards developed by the Internet Engineering Task Force (IETF) to support secure communication at the IP layer. Used widely to implement VPNs either as a full-fledged security gateway sitting between the corporate network and the router connected to the Internet, or as part of the router itself, IPSec and an associated suite of protocols offer complete cryptographic security.
There are three basic flavors of IPSec VPNs, each with an associated set of business requirements (Figure 1):
Remote-Access VPNs: These let individual users, such as telecommuters, connect to a corporate network. The user's laptop usually contains a VPN client that creates a secure tunnel to the security gateway at the corporate headquarters. Another flavor of this application is offered via creating an L2TP/PPTP session that is protected through IPSec.
Intranet VPNs: This type connects branch offices to the corporate headquarters, thus creating a transparent Intranet.
Extranet VPNs: These let companies connect with their business partners (for example, suppliers, customers, and joint ventures).
Figure 1: IPsec VPN applications
Basics of Cryptography
Cryptography is a set of mathematical functions that forms the foundation of all security protocols. With respect to IPSec, the following functions are important:
Encryption/Decryption: This forms the basis of confidentiality. Encryption is the process of taking a plaintext M and converting it to a ciphertext C of equal length. The ciphertext C is transmitted on the wire that protects the message M from any eavesdropper who overhears the communication. On the receiving end, decryption is performed that converts the ciphertext C back to plaintext M.
There are two kinds of cryptographic transforms available that differ on the usage of the key used to encrypt/decrypt:
Symmetric-Key Cryptography: Here only one shared key K is known to both the peers, but it is not known to any third party. Encryption is the function of two variables (M, K), and decryption is the function of (C, K). Many symmetric-key algorithms exist, but the popular ones are DES, 3DES, and AES. Most of these algorithms process one block of data at a time and transform it into a ciphertext block. These block-transform functions can be operated in different modes to add extra security. One such popular mode is cyclic block chaining (CBC), where each processed ciphertext block is added (in binary arithmetic) to the next plaintext block before it is input to the block transform function. An initialization vector (IV), random data of equal length, is used as the initial data for this mode. describes these algorithms and various modes in detail.
Asymmetric-Key Cryptography (also called public-key cryptography): Here a pair of keys is assigned per peer: a public key K1 that can be published and a private key K2 known only to the owner of the key pair. A text M encrypted by K1 can be decrypted only by K2 and vice versa, but not by any other means. The most popular public-key algorithm is RSA. Refer to Mohapatra for an introduction to public-key cryptography.
Note that symmetric key algorithms are computationally much faster than public key algorithms. For efficiency, symmetric-key algorithms like DES or 3DES are used for preserving confidentiality. Public-key algorithms are used in a "hybrid" or "key-management" mode to achieve authentication and non-repudiation.
Message Digest or Integrity Checksum Value (ICV): This forms the basis of data integrity. A message digest is a small "fingerprint" of a long message M that is obtained by applying a one-way hash function, H - a function that is easy to compute but for which it is computationally not feasible to find two messages M and M' such that H(M) = H(M'). If the message digest is included in the IPSec packet, the receiving end can compute the message digest on the received message and compare it with the included digest to find out if the message has been compromised. Popular hash functions are MD5 and SHA1.
Digital Signature: This forms the basis of non-repudiation. Typically a public-key algorithm, like RSA, is combined with a one-way hash function, like MD5 or SHA1. For a message M, a message digest H(M) is computed and encrypted (signed) by the sender's private key. This is called the digital signature for message M that is sent across the wire. Because only the owner knows the private key, the sender cannot repudiate the fact that he sent the message M.
Certificate: This forms the basis of authentication. Certificate is some data in a well-defined format that binds a person's identity to his/her public key and is signed by a "trusted authority." This certificate is "self-authenticating" because both the parties trust the "trusted authority" and have its public key available.
IPSec Modes of Operation
IPSec provides two different modes to exchange protected data across the different kinds of VPNs:
Transport Mode: This mode is applicable only for host-to-host security. Here protection extends to the payload of IP data. The IP addresses of the hosts must be public IP addresses.
Tunnel Mode: This mode is used to provide data security between two networks. It provides protection for the entire IP packet and is sent by adding an outer IP header corresponding to the two tunnel end-points. The unprotected packets generated by hosts travel through the protected "tunnel" created by the gateways on both ends. The outer IP header in Figure 2 corresponds to these gateways. Both intranet and extranet VPNs are enabled through this mode. Since tunnel mode hides the original IP header, it facilitates security of the networks with private IP address space.
Figure 2: IPsec modes of operationtunnel and transport
IPSec is an open, standards-based security architecture. The standards have defined the following concepts that are the building blocks of the architecture:
Security Association Database (SAD)
A security association (SA) is a unidirectional (simplex) logical connection between two IPSec systems, uniquely identifiable by a tuple. Figure 3 describes the SA data structure.
Figure 3: Security association structure
An SA indicates all the security services to the traffic carried by it (pointed to by the authentication transform, encryption transform, and the replay protectiona replay attack is a denial of service attack where an eavesdropper saves already traversed packets and sends them at a later point of time). It also has a lifetime associated with it, by the end of which the SA state becomes expired. An expired SA can no longer be used to protect traffic; a new SA for the same tuple has to be created. Because SAs are simplex, for bi-directional communication between two IPSec systems, there must be two SAs defined, one for each direction. SAs can be created either manually or through an automatic key-exchange protocol, IKE.
Security Parameter Index (SPI) is a 32-bit value used to identify an SA. The SPI is carried in the IPSec header of the protected packet and has local significance only. SAD contains all the SAs that are either created manually or automatically through negotiation, indexed by the tuple: .
Security Policy Database (SPD)
SPD contains what security services (policies) are to be offered to the IP traffic, classified by a set of fields of the IP packet, called a "selector." The selector for IPSec operations is the tuple
Figure 4: IPsec architecture
The "Policy Manager" module is the interface between the user adding a security policy and the SPD. The "IKE Daemon" module does the automatic SA negotiation between two IPSec peers. The "Certificate Manager" verifies and enrolls certificates for authentication purposes.
In short, a typical packet flow inside this architecture proceeds as follows:
A packet is received through the receive queue and passed to the IPSec packet processing module.
The IPSec packet processing module extracts the "selector" from the packet and looks up the SPD for a policy. If the policy is "discard", the packet is discarded. If the policy is "pass through", the forwarding engine forwards the packet normally.
If the policy is "IPSec", the SPD entry should point to an SA in SAD. The module then fetches the corresponding SAD entry and checks for validity. If the SA state is expired, the module indicates the IKE Daemon for another SA negotiation.
All the transforms depicted in the SA are performed on the packet with the help of the "cryptography" module.
The transformed packet is sent to the "transmit queue" for transmission.
IPSec standards have defined three main protocols: AH, ESP, IKE.
Authentication Header (AH)
AH provides data integrity and replay protection for the whole IP datagram and is an effective measure against IP-spoofing and session-hijacking attacks. AH uses the modified version of a hash function (such as, MD5 or SHA1) that takes an authentication key as input to compute the integrity checksum value (ICV). The ICV is included as part of the AH header. The receiver re-computes the ICV on the received packet and checks for equality. Any modifications that occurred to the packet during transmission can be discovered, as the ICVs will not match. The AH protocol specifies a set of mutable IP header fields that should not be used to compute the ICV (TOS, Fragment offset and flags, TTL, Checksum). Figure 5a illustrates the AH header.
Encapsulating Security Payload (ESP)
ESP provides data confidentiality, data integrity, and replay protection for the whole IP datagram. It uses a symmetric key algorithm (like 3DES-CBC or AES-CBC) to encrypt the data and, like AH, uses a secure hash algorithm to compute ICV. Unlike AH, the IP header is not protected against data-integrity attacks. Figure 5b illustrates the ESP header.
Figure 5: IPsec (a) AH format and (b) ESP format
Figure 6 shows how AH and ESP are used in tunnel mode to protect an IP packet.
Figure 6: AH and ESP in tunnel mode
Internet Key Exchange (IKE)
IKE defines the mechanism to establish SAs
required to secure the packets between the two IPSec peers. As
defined in Figure 3, the main components of an SA are
the transform details (the algorithm and the key) that will be
used to protect data. IKE defines an automatic and secure way
of negotiating these details between the two peers.
IKE uses the Internet Security Association and Key
Management Protocol (ISAKMP) as the framework and incorporates Oakley and
SKEME as the key-exchange protocol. All the parameter values to
be negotiated are defined in the IPSec domain of interpretation
(DOI) document. The IKE Daemon runs as an UDP application
with UDP port number 500. The protocol operates in two
Phase I (Authentication Phase)
When two peers over the Internet wish to communicate, it is
assumed that no secure channel exists. Therefore, the
objective of "phase I" is to establish a secure channel,
authenticate the negotiating parties, and generate shared
keys to protect IKE protocol messages. Two modes are available for use
in "phase I": main mode and aggressive mode. They both
accomplish the same thing"establishing an IKE SA that will
be used to protect the negotiating messages of phase II.
Aggressive mode is a little faster, but it does not provide
identity protection for the negotiating parties. Also, for
some authentication methods used (like the public key
encryption), aggressive mode is known to be vulnerable to
denial-of-service (DoS) attacks.
Main Mode Figure 7 shows the basic main-mode message
exchanges. In main mode, the negotiating parties use
six messages: the first two messages to negotiate the
security policy that will be used to protect the phase
II messages. The next two messages perform a
Diffie-Hellman key exchange and pass nonces (random
numbers sent for signing) to each other. The last two
messages are used to authenticate the peers.
There are four methods specified in the IKE standard to
authenticate the negotiating peers. They are:
Pre-Shared Keys (PSK): As the name
suggests, a shared secret is distributed out-of-band
to the peers. The peers use this information and
nonce parameters to create a hash that is used to
Digital Signatures (RSA or DSS): Here,
certificates of the peers are exchanged in the last
two messages and hashes are calculated over these
certificates to authenticate each other. There are
many protocols and standards available now that ease
the process of certificate enrollment, certificate
request, and certificate status checking. Some
popular ones are RSA Labs' PKCS #7, PKCS #10, Cisco's
Simple Certificate Enrolment Protocol (SCEP), and
Online Certificate Status Protocol (OCSP).
RSA Public Key Encryption: With this, each
negotiating party has a public-private key-pair that
they use to encrypt/decrypt messages starting from
message #3. Nonces exchanged are secured through this
and the parties calculate hashes over these nonces to
authenticate each other.
Revised RSA Public Key Encryption: This
method is similar to the prior method, but reduces
the number of public key operations from four to two
and, instead, incorporates two symmetric-key
The authentication data payload shown in Figure
7 is different for each of the previously described modes. For
example, for PSK mode the previously created hash is used. For an RSA digital-signature
mode, the certificate and the digital signature are
sent as authentication data.
Figure 7: IKE main mode message exchange
In aggressive mode, you need only three messages to
establish the IKE SA. However, the identities of the
parties involved are revealed. Figure 8
illustrates aggressive mode exchanges.
Figure 8: IKE aggressive mode message exchange
Phase II (Key Exchange)
Phase II, also called as the "Quick Mode," is used to
establish the IPSec SA and to generate new keying material. A
full Diffie-Hellman key exchange may be done to provide
perfect forward secrecy (PFS), otherwise the keys are derived
from the phase I keying material. The basic quick-mode
message exchanges are illustrated in Figure 9.
Figure 9: IKE quick mode message exchange
IPsec Packet Processing
The following two flowcharts describe the IPSec operation on the security device for inbound and outbound packets.
Inbound Packet Processing
Figure 10 shows the functional flowchart for inbound packets, in other words, the protected packets that arrive at the security gateway. These are the packets that typically come from the public network, like the Internet to the private network. The packets have to be decoded (decrypted), authenticated, and forwarded to the private network. As the selector for the packet goes deep into the IP protocol fields, such as the port numbers, the packet may have to go through the IP reassembly process before the IPSec processing starts.
Outbound Packet Processing
Figure 11 shows the functional flowchart for packets that arrive from the private network, destined to another private network across the Internet. These packets need to be protected. This process does affect the MTU, as it adds more bytes with headers, so the packets may have to be fragmented after the IPSec processing. IPSec is not affected by fragmentation, but the IPSec security device does have to participate in the PMTU discovery process because it inserts extra headers.
IPSec and NAT Interaction
IPSec and NAT are inherently not compatible protocols, as IPSec protects the packet's integrity, whereas NAT, as a protocol, changes the IP header and TCP/UDP header. If both IPSec and NAT operations are supported in the same security device, then the problem can be avoided by performing the NAT operation before doing IPSec and making sure that the IPSec end-points are in the public address space.
For scenarios where there is a NAT device in-between (NAT-in-the-middle problem), the IPSec standards group at IETF has proposed a new protocol called "NAT traversal (NAT-T)". Here, the IPSec packet is encapsulated within a UDP packet using the IKE UDP port 500. Negotiation of NAT-T support of the peers as well as detection of NAT presence in the path is done during IKE phase-I.
Engineering A High-Performance Security Gateway
Encryption and authentication are extremely compute-intensive functions. For example, a particular DES implementation in C optimized for the Pentium Pro requires 456 cycles to encrypt a single 64-bit block. This only corresponds to a speed of 28 Mbps on a 200-MHz Pentium platform. Phil Karn has written highly tuned assembly code for the Intel Pentium processors that achieves around 24 Mbps with 3DES on a Pentium II running at 400 MHz. A security gateway that must perform at wire speed with 64-byte packets cannot scale much beyond this throughput range with a software-only implementation. Specialized ASICs that perform the cryptographic functions, including encryption and authentication, are called crypto accelerators. These devices are necessary to scale to higher throughput rates. There are different types of crypto accelerators available in the market. At this point of time, these crypto accelerators seem to fall into three general categories:
Processors with Basic Algorithm Support These processors perform basic symmetric-key operations such as DES, 3DES, and AES, and hash operations such as SHA1, MD5, and others.
Packet Processors These take in a packet along with an SA and do the complete packet processing (for example, the addition of the AH or ESP header, as required) in addition to supporting the prior functionality.
Inline Security Coprocessors These processors can handle SA lookup and packet handling, as well as SPD verification.
You can choose the right kind of crypto accelerators with the right price/performance as needed for your system. For example, if you are building a 1-Gbps security gateway and want to offload all of the security processing from your main network processor, you should use an inline processor.
IPSec is a thorough and complete solution for protecting IP traffic. IPSec protects all traffic against unauthorized modification and eavesdropping, and also securely authenticates the communicating parties. Because of the cost-savings potential of IPSec VPNs, along with general interest in security, the VPN market is still growing strongly despite a nearly two-year-old economic downturn. Infonetics Research has forecasted end-user VPN product and service expenditures to grow 117%, from $21.3 billion to $46.2 billion between 2002 and 2006. However, certain problem areas that need to be addressed are as follows.
The protocol is as strong as the underlying algorithms it employs, so proper algorithm selection is important to network security. Furthermore, for smooth operation, IPSec requires a public-key infrastructure (PKI). Such infrastructures are still in their infancy, and wide-scale key infrastructures are just emerging on the Internet. All-in-all, the management of the security and access policies is a complicated field and a thorough knowledge of several concepts is necessary to properly administer these policies. Finally, as IPSec and IKE are evolving standards, interoperability between different vendors is still a problem.
Still, the overall impact of IPSec will be tremendous in the years to come. IPSec makes it possible to securely connect offices, users, and partners to the network and safely transmit information as a very cost-effective solution. IPSec does this in a manner completely transparent to the end users. As the networks migrate to Ipv6, IPSec will become an integral component of those networks as well.