The ESP Protocol

A secure Transport Protocol must be used to exchange the encrypted traffic between Peers once parameters and key material have become available. Two options have been defined for use with IPSEC. The first being the Authentication Header Protocol ( or AH ) and the second being the Encapsulating Security Payload Protocol ( or ESP ). Some details unique to the AH protocol prevent it from being used when a device performing Network Address Translation ( or NAT ) exists between two Peers.

Both Transport Protocols offer two different modes of operation. These are referred to as Transport and Tunnel mode. Transport mode is used to protect the data contained within an IP packet payload. Tunnel mode is used to protect an entire IP datagram by encrypting the original header along with the payload data. This encrypted data is then encapsulated in a new IP datagram using header information that is suitable for public network routing. Since Tunnel mode retains the original IP header information, it can be used to process network traffic on behalf of other hosts. This allows an IPSEC Peer to function as a security gateway by encrypting and encapsulating all traffic that matches a security policy and then forwarding the protected traffic to an appropriate peer gateway. The peer gateway then decapsulates and decrypts the traffic which can then be routed based on the original IP header information.