About IPSec Algorithms and Protocols

IPSec is a collection of cryptography-based services and security protocols that protect communication between devices that send traffic through an untrusted network. Because IPSec is built on a collection of widely known protocols and algorithms, you can create an IPSec VPN between your XTM device and many other devices that support these standard protocols. The protocols and algorithms used by IPSec are discussed in the subsequent sections.

Encryption Algorithms

Encryption algorithms protect the data so it cannot be read by a third-party while in transit. Fireware XTM supports three encryption algorithms:

Authentication Algorithms

Authentication algorithms verify the data integrity and authenticity of a message. Fireware XTM supports three authentication algorithms:

HMAC-MD5 (Hash Message Authentication Code — Message Digest Algorithm 5)

MD5 produces a 128 bit (16 byte) message digest, which makes it faster than SHA1 or SHA2. This is the least secure algorithm.

HMAC-SHA1 (Hash Message Authentication Code — Secure Hash Algorithm 1)

SHA1 produces a 160-bit (20 byte) message digest. Although slower than MD5, this larger digest size makes it stronger against brute force attacks.

HMAC-SHA2 (Hash Message Authentication Code — Secure Hash Algorithm 2)

Fireware XTM 11.8 and higher supports three variants of SHA2 with different message digest lengths.

SHA2 is stronger than either SHA1 or MD5. Because SHA2 requires more computational resources, it is supported only on XTM devices with hardware cryptographic acceleration for SHA2.

SHA2 is not supported on XTM 510, 520, 530, 515, 525, 535, 545, 810, 820, 830, 1050, and 2050 devices. The hardware cryptographic acceleration in those models does not support SHA2.

IKE Protocol

Defined in RFC2409, IKE (Internet Key Exchange) is a protocol used to set up security associations for IPSec. These security associations establish shared session secrets from which keys are derived for encryption of tunneled data. IKE is also used to authenticate the two IPSec peers.

Diffie-Hellman Key Exchange Algorithm

The Diffie-Hellman (DH) key exchange algorithm is a method used to make a shared encryption key available to two entities without an exchange of the key. The encryption key for the two devices is used as a symmetric key for encrypting data. Only the two parties involved in the DH key exchange can deduce the shared key, and the key is never sent over the wire.

A Diffie-Hellman key group is a group of integers used for the Diffie-Hellman key exchange. Fireware XTM can use DH groups 1, 2, 5, 14, 15, 19, and 20.

For more information, see About Diffie-Hellman Groups.


Defined in RFC 2402, AH  (Authentication Header) is a protocol that you can use in manual BOVPN Phase 2 VPN negotiations. To provide security, AH adds authentication information to the IP datagram. Most VPN tunnels do not use AH because it does not provide encryption.


Defined in RFC 2406, ESP (Encapsulating Security Payload) provides authentication and encryption of data. ESP takes the original payload of a data packet and replaces it with encrypted data. It adds integrity checks to make sure that the data is not altered in transit, and that the data came from the proper source. We recommend that you use ESP in BOVPN Phase 2 negotiations because ESP is more secure than AH. Mobile VPN with IPSec always uses ESP.

See Also

About IPSec VPNs

Give Us Feedback  •   Get Support  •   All Product Documentation  •   Knowledge Base