IPSec Packet Types
IPSec defines a set of headers that are added to IP packets. These new headers are placed after the IP header and before the Layer 4 protocol. They provide information for securing the payload of the IP packet. The security services are provided by the Authentication Header (AH) and the Encapsulating Security Payload (ESP) protocols. AH and ESP can be used independently or together, although for most applications, just one of them is sufficient. For both of these protocols, IPSec does not define the specific security algorithms to use; instead it provides an open framework for implementing industry−standard algorithms.
Authentication Header (AH), described in RFC 2402, ensures the integrity and authenticity of the data, including the invariant fields in the outer IP header. It does not provide confidentiality protection, meaning it does not provide encryption. When an AH mode header is added to an IP packet, it provides authentication for as much of the IP header as possible, as well as for all the upper−layer protocols of an IP packet. However, some of the IP header fields may change in transit, and the value of these fields, when the packet arrives at the receiver, may not be predictable by the sender. These fields are known as mutable fields, and their values cannot be protected by AH.
Predictable fields are also known as immutable fields.
AH provides authentication and integrity to packets by performing an integrity check, which is a keyed hash using a shared secret value that creates a message digest, when used together they are known as the integrity check value (ICV). The ICV is computed on the following:
IP header fields that are either immutable in transit or predictable in value upon arrival at the end point for the Authentication Header security association.
The Authenticated Header, payload length, reserved fields, SPI, sequence number, authentication data, and padding bytes.
The upper−layer protocol data.
The result of the integrity check value helps the remote peer to determine if the packet has changed in transit. Upon receipt of the packet, the remote peer performs the same integrity check on the packet and compares the integrity check value that it has computed against the value of the integrity check value that the sender provided. The following are immutable fields and are included in the Authentication Header integrity check value computation:
Internet header length
AH cannot include in the integrity check any mutable field of the packet that might be modified by other routers within the transmission path of the packet from the source to the destination. The following are mutable fields and are not included in the computation of the AH ICV:
Type of Service (ToS)
Encapsulating Security Payload
Encapsulating Security Payload (ESP), described in RFC 2406, ensures the confidentiality, integrity, and optional authenticity of the data, yet ESP does not use the invariant fields in the IP header to validate data integrity. ESP has an optional field used for authentication. It contains an ICV that is computed over the remaining part of the ESP, minus the authentication field. The length of the optional field varies depending on the authentication algorithm that is chosen. If authentication is not chosen, the ICV is omitted. Authentication is always calculated after the encryption is done.
ESP performs encryption at the IP packet layer. It supports a variety of symmetric encryption algorithms. The default algorithm for IPSec is the 56−bit Data Encryption Standard using the cipher block chaining mode transform (DES−CBC). This cipher must be implemented to guarantee interoperability with other IPSec products.
The services that are provided by ESP depend on the options that are configured during the IPSec implementation and after an IPSec SA is established.
IPSec Modes of Operation
The format of the AH and ESP headers and the values contained within each packet vary according to the mode in which each is used. IPSec operates in either tunnel or transport mode.
Transport mode is used when both peers are hosts. It may also be used when one peer is a host and the other is a gateway if that gateway is acting as a host. Transport mode has an advantage of adding only a few bytes to the header of each packet. When transport mode is used, the original header is not protected. This setup allows the true source and destination addresses to be viewed by intermediate devices implemented based on the contents of the IP header. One advantage of not changing the original header is that Quality of Service (QoS), can be processed from the information in the IP header. One disadvantage is that it is possible to do traffic analysis on the packets. Transport mode can be used only if the two end devices are the ones providing IPSec protection. Transport mode cannot be used if an intermediate device, such as a router or firewall, is providing the IPSec protection. Figure 6.1 displays an example of a normal IP packet.
Figure 6.1: IP packet.
When Authentication Header (AH) is used in transport mode, the AH services protect the external IP header along with the data payload. It protects all the fields in the header that do not change in transport. The AH header goes after the IP header and, if present before the ESP header, if present, and other higher−layer protocols, like TCP. Figure 6.2 shows an example of using AH in transport mode.
Figure 6.2: AH in transport mode.
When ESP is used in transport mode, the IP payload is encrypted; however, the original headders are not encrypted. The ESP header is inserted after the IP header and before the upper−layer protocol header, like TCP. The upper−layer protocols are encrypted and authenticated along with the ESP header. ESP does not authenticate the IP header or the higher−layer information, such as TCP port numbers in the Layer 4 header. Figure 6.3 shows an example of using ESP in transport mode.
Figure 6.3: ESP in transport mode.
Tunnel mode is used between two gateway devices, such as two PIX firewalls, or between a host and a gateway. In tunnel mode, the entire original IP packet is encrypted and becomes the payload of a new IP packet. The new IP header has the destination address of its IPSec peer. This allows for tunneling of IP packets from a protected host through a router or firewall usually to another router or firewall, which can both be acting as security gateways. One of the advantages of tunnel mode is that intermediate devices, such as routers, can do encryption without any modification to the end system. All the information from the original packet, including the headers, is protected.
Tunnel mode protects against traffic analysis because, although the IPSec tunnel end points can be determined, the true source and destination end points cannot be determined because the information in the original IP header has been encrypted.
When Authentication Header (AH) is used in tunnel mode, the original header is authenticated and the new IP header is protected in exactly the same manner it was protected when used in transport mode. Figure 6.4 shows an example of using AH in tunnel mode.
Figure 6.4: AH in tunnel mode.
When ESP is used in tunnel mode, the original IP header is protected because the entire original IP packet is encrypted. When both authentication and encryption are configured, encryption is performed first, before authentication. The reason for this process to take place in this order is that it facilitates rapid detection and rejection of replayed or bogus packets by the receiving node. Prior to decrypting the packet, the receiver can detect the problem and potentially reduce the impact of an attack. Figure 6.5 shows an example of using ESP in tunnel mode.
Figure 6.5: ESP in tunnel mode.
When you want to make sure that certain data from a known and trusted source gets transferred with integrity and the data does not need confidentiality, use the AH protocol. AH protects the upper−layer protocols and the IP header fields that do not change in transit, such as the source and destination addresses. AH cannot protect those fields that change in transit, such as the Type of Service (TOS) field. When the fields are protected, the values cannot be changed without detection, so the IPSec node will reject any altered IP packet. In summary, AH does not protect against someone sniffing the wire and seeing the headers and data. However, because headers and data
cannot be changed without the change being detected, changed packets would get rejected.
If data confidentiality is needed, use ESP. ESP will encrypt the upper−layer protocols in transport mode and the entire original IP packet in tunnel mode so that neither is readable. ESP can also provide authentication for the packets. However, when you use ESP in transport mode, the outer IP original header is not protected; in tunnel mode, the new IP header is not protected. You will probably implement tunnel mode more than transport mode during initial IPSec usage. This mode allows a network device, such as a router to act as an IPSec proxy.