Last updated at Wed, 13 Dec 2017 18:51:48 GMT


While writing the blog “Basics of IPsec“, I gave an overview of some of concepts embedded in IPsec. For any of you, who are curious about the details, this article and the succession ones will act as a guide. The purpose of this article is to gain knowledge regarding concepts of IPsec Authentication Header.


Starting with the protocols, we have already discussed that IPsec operates using two security protocols. We can have Authentication Header, AH and Encapsulating Security Payload, ESP, either of which can be selected during Phase 2 of Tunnel Negotiation.

Authentication Header

Authentication Header (AH) is applied to give a connectionless integrity and authentication of the data origin for IP datagrams. Also if replay attacks are a concern, there is an optional service available to be opted in AH by the receiver, so as to provide protection against them. This is implemented during the Security Association (SA).

Level of Authentication

AH gives authentication to the IP header. But in some cases, the header field and their values get altered. When the receiver gets the packet, there are changes in the original packet sent. Thus, this protocol doesn’t give complete protection to the packet. It only lets the receiver know whether the message received is not tempered with, but does nothing to protect the message itself.

How is Authentication Done?

The algorithm to check the authenticity of the data packets received is carried out by running a checksum on the packet. The checksum is calculated through HMAC (Hash Message Authentication Code) using a secret key that is only known to the sender and the recipient. HMAC uses a cryptographic hash function and a secret cryptographic key. The hash functions applied can be either MD5 or SHA-1.

Message Digest 5 (MD5)

Also known as digital signature or message digest, this algorithm produces a 128-bit hash and a 16-byte key. The input message is broken into 32-bit words and then padded (length-wise) to make it divisible by 512. The four-word buffers compute the message digest.

Secure Hash Algorithm (SHA-1)

This algorithm produces a 160-bit hash and a 20-byte key. This is comparatively secure because of larger hash it produces. It produces message digest similar to MD-5

The length of the message is variable for both cases.


There are two protocol header’s available for the version used that proceed the AH header. The value is to be set 51 for Protocol (IPv4) or Next Header (IPv6, IPv6 Extension) fields. The fields that make up our AH are:

****Next Header:

A byte long field giving the type of next payload. If this value is 4 so IPv4, 41 is for IPv6 and 6 tells that it is TCP. This is used to link headers together.

****Payload Length:

A byte long field measuring the total length of AH in 4-byte units minus 2. So suppose if the integrity algorithm gives a value of 96-bit, this field will be 4. (3 32-bit words fixed field plus 3 32-bit words for ICV, minus 2)


These two bytes have been reserved for future implementation. So when formulating the packet it is set to be zero and is ignored at the receiver end.

****Security Parameter Index:

A 32-bit field which is used by the receiver to identify the SA of the incoming packet.

****Sequence Number:

An unsigned 32-bit field. This is basically packet counter which is incremented by one for each packet sent. At the initialization of SA, both sender and receiver Sequence Number is set to zero

****Extended Sequence Number:

This is an extension that is available for high-speed IPsec implementations. For IKEv2, this negotiation is implicit.

****Integrity Check Value:

The last field is a variable-length which contains ICV for the packet. ICV is basically the results obtained from the hashing algorithm. The value must be an integral multiple of 32 bits in length.

AH with Modes

Now that we know about its format, the last thing we would like to touch is the effective working of AH in both modes available.

Transport Mode:

Transport Mode provides protection from end-to-end conversations between two hosts. This is not a tunneling procedure but a secured IP connection. The IP packet is changed by inserting the AH header between the IP header and payload. There is shuffling of protocol code linking headers together. When the packet is authenticated, the original protocol type (UDP, TCP) is restored. This is done by removing the AH header and replacing it with the saved Next Protocol.

Tunnel Mode:

This mode makes use of the tunneling concept. As the Tunnel Mode encapsulates the entire IP packet (IP packet + Payload), the source and destination are different from the original packet. When this packet is received at its destination, it discards its IP and AH headers thus giving back the original IP datagram.


The following line of code is run on the CLI to define the Authentication Header using md5 as Hashing Algorithm.

set security ike proposal IKE-PRO authentication-algorithm md5

As we said earlier, we can use either md5 or sha-1. We can also add flavors of them with HMAC. Here is how;

set security ipsec proposal IPSEC-PRO authentication-algorithm hmac-md5-96

Note: Since JUNOS is case sensitive therefore you have got to take care of the upper and lower cases.

This is how we will work out in Cisco, we need to define an ISAKMP policy and under its hierarchy mentions our HASH i.e. hash sha512; so it is going to look like this;

crypto transform-set someset ah-md5-hmac esp-des 
crypto isakmp policy 10 
hash sha512