Why IPsec matters
Devices exchange information across networks by sending small units of data called IP packets. When that traffic crosses the public internet or another untrusted network, organizations need a way to protect it from interception and tampering. IPsec provides security at the network layer, so it can protect traffic from multiple applications without requiring each application to implement its own protection.
IPsec can provide four important safeguards:
- Confidentiality: Encryption helps prevent someone observing network traffic from reading protected data.
- Integrity: Cryptographic checks help recipients detect whether protected data changed in transit.
- Authentication: Communicating peers verify that they’re establishing a connection with an authorized device or gateway.
- Anti-replay protection: Sequence numbers help identify packets that an attacker captures and tries to resend.
These safeguards support broader network security practices. They can reduce some risks associated with man-in-the-middle attacks (MITM), but they can’t protect a compromised endpoint or prevent every misuse of an authorized connection.
How IPsec works
IPsec protects traffic according to rules defined by administrators. In a typical encrypted IPsec VPN, two peers, such as firewalls or VPN gateways, agree on how to secure traffic before exchanging protected packets. The process usually follows five steps:
1: Identify traffic to protect
Security policies determine which packets require IPsec, which may pass normally, and which should be blocked.
2: Negotiate a secure connection
Internet Key Exchange version 2 (IKEv2) helps peers authenticate each other, agree on cryptographic settings, and establish keys.
3: Create security associations
These agreements specify the settings each peer uses to protect or process traffic in a particular direction.
4: Protect and transmit packets
Encapsulating Security Payload (ESP) applies the agreed protection, commonly including encryption and integrity checks, before packets cross the network.
5: Verify incoming packets
The receiving peer checks the relevant protection, rejects packets that fail validation, and processes the original traffic.
For example, two office gateways can use IKEv2 to establish the connection and ESP to protect traffic between them. The negotiation sets up the protection, but doesn’t repeat for each individual packet. Understanding data encryption helps explain why intercepted ciphertext isn’t the same as readable information.
Key IPsec components and modes
IPsec is a suite rather than a single protocol. Different components handle packet protection and key management, while operating modes determine which parts of a packet are covered.
The main IPsec protocols
- ESP (Encapsulating Security Payload): Commonly used to encrypt data and provide integrity, authentication, and anti-replay protection. It’s central to most encrypted IPsec VPNs.
- AH (Authentication Header): Provides integrity and data-origin authentication, but not encryption. It’s less commonly used because ESP meets most practical requirements.
- IKEv2 (Internet Key Exchange): Helps peers authenticate, negotiate security associations, and manage cryptographic keys. It supports the establishment and maintenance of secure IPsec connections.
Transport mode vs. tunnel mode
Transport mode primarily protects the information carried inside an IP packet while retaining the original IP header. It’s commonly associated with host-to-host communication.
Tunnel mode protects the entire original IP packet by placing it inside a new packet with an outer IP header. This is the usual choice for site-to-site VPNs and many remote-access VPNs.
The distinction matters because tunnel mode can protect the original addressing information inside the encrypted packet, while the new outer header still provides information needed to route traffic. Neither mode automatically makes the endpoints themselves trustworthy.
Common IPsec use cases and examples
IPsec is often associated with VPNs, but it can also protect communications directly between hosts. The right deployment depends on which devices communicate and where the security boundary sits.
Site-to-site VPNs
A company with offices in different cities can connect its network gateways over the internet. With IPsec tunnel mode, traffic between the two locations travels through a protected connection without requiring employees to secure each application separately.
Remote-access VPNs
An employee working outside the office can use an IPsec-based VPN client to establish a secure connection to an organizational gateway. The VPN protects traffic crossing that connection, but the organization still needs access controls and secure devices.
Cloud and hybrid connectivity
An organization can connect an on-premises network to a cloud network using an IPsec tunnel. That connection protects traffic in transit between the two environments, and is one part of cloud network security, alongside segmentation, identity controls, and secure configuration.
How IPsec fits into security operations
IPsec is a preventive network control, helping to protect traffic moving between approved peers. Security teams, however, still need to monitor connections, maintain gateways, and investigate unusual activity. Encryption doesn’t replace visibility or detection.
Important operational practices include:
- Review connection policies: Confirm which networks and devices are permitted to communicate, and restrict access only to what’s needed.
- Maintain gateways and clients: Apply updates, use supported cryptographic settings, and investigate known weaknesses. Network vulnerability scanning is relevant to identifying weaknesses in network-facing systems.
- Monitor connection activity: Look for unexpected peers, repeated authentication failures, tunnel interruptions, and unusual traffic volumes.
- Correlate network signals: Analyze gateway logs, flow records, and other telemetry to understand suspicious connections. Network traffic analysis (NTA) can support this work, although encrypted payloads may not be readable to a network sensor.
IPsec also has boundaries, as it’s able to protect selected traffic while it crosses a connection, not necessarily after that traffic reaches its destination. Security operations teams combine it with endpoint protection, segmentation, identity and access management (IAM), and incident response (IR) to address risks that encryption alone cannot resolve.
Author
