Checkpoint FW-1 VPN Connectivity with Nortel Contivity

Steve Luke Atos Origin

14th February 2003

CP FW-1 to Nortel Contivity VPN Configuration - 2 -

Version Description 0.1 Draft – Steve Luke 0.2 Initial Proof Release – Steve Luke 1.0 Official Release (Added Appendix / TOC) – Steve Luke

CP FW-1 to Nortel Contivity VPN Configuration - 3 -

Table of Contents

Table of Contents..............................................................................................3 Introduction......................................................................................................4 Test Environment..............................................................................................4 Checkpoint FW-1 Configuration ..........................................................................5 Using NAT with the VPN ..................................................................................11 Nortel Contivity Configuration ..........................................................................12 Appendix A: Troubleshooting / Issues Experienced...........................................18

CP FW-1 to Nortel Contivity VPN Configuration - 4 -

Introduction This document defines the configurations needed for site-to-site VPN connectivity between a Checkpoint FW-1 firewall and a Nortel Contivity Extranet VPN switch. Test Environment Checkpoint Sun Netra T1 UltraSparc IIi CPU 440Mhz 512Mb RAM Solaris 2.6 (Latest Patches per DIBBS) Checkpoint FW-1 ver 4.1 SP6 FM License with 3DES Encryption Nortel Contivity Nortel Contivity 600 Switch Contivity OS ver V03_60.45 BIOS P03 Celeron 300 Mhz 128Mb RAM 2Gb HDD Schematic

VPN Tunnel

Checkpoint FW-1 Nortel Contivity

Test PC 1 Test PC 2

LAN LAN

CP FW-1 to Nortel Contivity VPN Configuration - 5 -

Checkpoint FW-1 Configuration The following are screenshots and descriptions of the configuration necessary to create a VPN tunnel between a Checkpoint FW-1 firewall and a Nortel Contivity Extranet Switch. Objects needed to be created on the CP firewall.

• CP FW Object (Naturally!!!) • Nortel Contivity Object • Network Object for Local Network (Encryption Domain) • Network Object for Remote Network (Encryption Domain)

Other Settings

• Encryption tab of the Policy Properties • Key Exchange Rules • Encryption Rules

CP FW Object The checkpoint object should have the external IP address as the main Interface address on the ‘General’ tab of the object properties. It should also be licensed to this address. Although this is not essential for some operation, it has been see in test and production environments to cause connectivity issues for VPN’s and Client VPN’s. The important settings for this object are the VPN tab settings:

CP FW-1 to Nortel Contivity VPN Configuration - 6 -

Click Edit under the ‘Encryption Schemes Defined’

It does not matter for this screen, if you have more than one option selected (ie. DES + CAST + 3DES, or MD5 + SHA1) as long as DES is selected, MD5 and Support keys….. Nortel Contivity Object This object should be set as an External / Gateway on the General tab as follows:

CP FW-1 to Nortel Contivity VPN Configuration - 7 -

Go to the VPN tab:

CP FW-1 to Nortel Contivity VPN Configuration - 8 -

Click Edit:

Network Objects You should create a network object for each of the 2 encryption domains. The size of the local encryption domain subnet is critical, this is the network object you are going to specify in the CP FW Object in the VPN tab. If you already use a group object with many networks in it as your encryption domain, then the network object in that group in which the connections will be initiated from must be correct. If this is not the same as what is specified in the Branch Connection Static Routing setting on the Contivity, this will not work. The remote network object should be specified as the domain in the Contivity FW Object (VPN tab)

CP FW-1 to Nortel Contivity VPN Configuration - 9 -

Encryption tab of the Policy Properties This is where you specify the IKE / Rekey timeouts for the connection. As noted before this is not essential, but preferable for smooth operation.

CP FW-1 to Nortel Contivity VPN Configuration - 10 -

Key Exchange Rules These are the rules that specify how the Initial VPN key exchange and setup is performed between the 2 firewalls.

Encryption Rules These rules specify the traffic that should be encrypted and sent down the tunnel.

Right Click on ‘Encrypt’ and select ‘Edit Properties’. Here you will specify how the traffic is encrypted.

CP FW-1 to Nortel Contivity VPN Configuration - 11 -

Using NAT with the VPN If you need to use NAT on the Checkpoint side, so that the Nortel side does not see your private IP addresses, the following should be done:

1. A NAT rule should be created in the NAT Rulebase. 2. Both the real network and NATted subnet range should be included in the encryption

domain group. 3. The ‘Branch Connection Static Routing’ on the Nortel Contivity should be configured

with the NATted subnet range. Conclusion That concludes the configuration of the Checkpoint FW-1. You should now push the policy and bounce the FW (fwstop ; fwstart)

CP FW-1 to Nortel Contivity VPN Configuration - 12 -

Nortel Contivity Configuration The configuration of the Contivity is broken down into these parts:

• Creation of the local network (Nortel side) • Branch Office Connection setup

Creation of the local network is as follows (if this is not already done)

1. Go to Profiles – Networks 2. Enter a name and click ‘Create’ 3. Edit the properties as below:

4. Close

CP FW-1 to Nortel Contivity VPN Configuration - 13 -

Next go to Profiles – Branch Office

1. Click Add Group 2. Enter a group name

3. OK the name 4. Click ‘Edit’ next to the new Group you have created. Then configure the options as in

the picture below.

CP FW-1 to Nortel Contivity VPN Configuration - 14 -

5. Use the configure button next to each group to configure it as displayed above. 6. OK these changes. 7. Back on the groups page, click ‘Define Branch Office Connection’ 8. Select the group you just made and enter a name for the connection

CP FW-1 to Nortel Contivity VPN Configuration - 15 -

9. OK the new connection 10. Click Edit next to the new connection

CP FW-1 to Nortel Contivity VPN Configuration - 16 -

11. Fill in the details, including Local and Remote firewall external IP addresses 12. Click ‘IP’ under ‘Configure Routing’ 13. Add the local network (Nortel side) that you previously created 14. Add the remote network (CP side) *** This is the encryption domain on the CP side

and must be exactly the same size as the specified encryption domain on the CP FW Object.

CP FW-1 to Nortel Contivity VPN Configuration - 17 -

15. OK this and go back to the Branch Office Connection page. 16. Enter the Pre-Shared Secret in the box provided and OK this. 17. You should now be able to use the ‘Test’ button on the Branch Connection page and

see a successful tunnel establishment.

CP FW-1 to Nortel Contivity VPN Configuration - 18 -

Appendix A: Troubleshooting / Issues Experienced Replication On initial build of the test systems, using a standard VPN type configuration on both firewalls, I received the same error messages as seen in the production environment. These errors were: Invalid ID Information This is caused by a subnet size mismatch in the encryption domain of the CP Firewall, and the Branch Office Static Routing section of the Contivity. If the subnet in the domain settings of the Checkpoint FW object is of a different size to the network specified in the Static Routing section of the Contivity, Phase 1 of the negotiation will complete, but at Phase 2 Stage 1, the negotiation breaks down and the Contivity returns and error to the Checkpoint FW. This is represented in the firewall log as: 11:24:50 keyinst x.x.x.x >daemon src x.x.x.x dst x.x.x.x IKE Log: Phase 1 completion. DES/MD5/Pre shared secrets Negotiation Id: 471f77ee89fc09ef-7a469014265b7bc8 11:24:50 keyinst x.x.x.x >daemon src x.x.x.x dst x.x.x.x IKE Log: Received Notification from Peer: invalid id information Negotiation Id: fbded2df This shows in the Contivity System log as: 11:21:29 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x] attempting login 11:21:29 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:26 authenticated using LOCAL 11:21:29 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:26 bound to group /Base/VPN_Test/CP-VPN 11:21:29 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:26 authorized 11:21:29 tEvtLgMgr 0 : Security [11] Session: network IPSEC[x.x.x.x-x.x.x.x] attempting login *11:21:29 tEvtLgMgr 0 : ISAKMP [13] Invalid ID information in message from x.x.x.x The caveat here is that this problem does not show if the connection is initiated from the Contivity (via the Test VPN option). However, even though both Phase 1 and 2 complete successfully and the tunnel appears to be up, traffic will still not flow. NB: This can also occur if ‘Support Key Exchange for Subnets’ is disabled on the CP FW-1. No Proposal Chosen

CP FW-1 to Nortel Contivity VPN Configuration - 19 -

This error is because of a mismatch in the encryption and authentication settings between the 2 firewalls. During the tests all options were tried and the error only occurred with different settings This appears in the CP FW log as: 12:15:30 keyinst x.x.x.x >daemon src x.x.x.x dst x.x.x.x IKE Log: Received Notification from Peer: no proposal chosen It appears in the Contivity System log as: 12:12:38 tEvtLgMgr 0 : ISAKMP [13] No proposal chosen in message from x.x.x.x This mismatch can occur because the settings are different for:

• Encryption Scheme (IKE, FWZ etc) • Key Exchange Encryption (DES, CAST, 3DES) • Traffic Encryption (DES, DES-40, 3DES) • Data Integrity (MD5, SHA1) • Transform (ESP, AH) • Perfect Forward Secrecy setting*

*The PFS setting also produces some irregular results: PFS Setting CP Nortel Result 1 On On PFS used, tunnel works 2 Off On Does not work (No Proposal Chosen) 3 On Off PFS used, tunnel works (Should not work) 4 Off Off Tunnel works without PFS For the exact settings needed to synchronise both firewalls please read the next section on configuration. Invalid cookie This is a very sporadic error, that only occurred twice in production and twice in test. There are 2 possibilities as to why this happens, a possible error with PFS, or with Rekey timeouts, both however, are issues relating to the re-establishment of a previously established tunnel, having changed configuration. This issue does not occur on FW-FW VPN’ s, only with FW-Other VPN’ s. Checkpoint believe this to be a possible error with encryption domains (https://support.checkpoint.com/advanced/idsearch.jsp?id=sk6468&QueryText=%28%22invalid%22+AND+%22cookie%22%29&) , however this was not the case here. It was an error that appeared after a change to the Nortel. The two solutions I have found are to bounce the firewall (fwstop ; fwstart) or to delete the SA’ s from the state tables. The second solution however, only applies if the policy was pushed before the last tunnel establishment, as a policy push clears the state table anyway, and backs it up to an ‘old state table’ which the kernel references for previous connections. This is not an error relating to our tests, just an error that appears with certain configuration changes that need a firewall bounce. Successful Operation A successful VPN tunnel establishment should appear in the FW log as:

CP FW-1 to Nortel Contivity VPN Configuration - 20 -

15:01:06 keyinst x.x.x.x >daemon src x.x.x.x dst x.x.x.x IKE Log: Phase 1 completion. DES/MD5/Pre shared secrets Negotiation Id: 7da52c0da9411d43-fc46193060c62f51 15:01:06 keyinst x.x.x.x >daemon proto ip src x.x.x.x dst x.x.x.x srckeyid 0xc1a32fe2 dstkeyid 0x000f4d8e rule 0 scheme: IKE methods: Combined ESP: DES + MD5 + PFS (phase 2 completion) for subnet: x.x.x.x (mask= 255.255.255.0) and for subnet: x.x.x.x (mask= 255.255.255.0) In the Contivity System Log, this appears as: 16:24:28 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x] attempting login 16:24:28 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:12 authenticated using LOCAL 16:24:28 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:12 bound to group /Base/VPN_Test/CP-VPN 16:24:28 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:12 authorized 16:24:28 tEvtLgMgr 0 : Security [11] Session: network IPSEC[x.x.x.x-255.255.0.0] attempting login 16:24:28 tEvtLgMgr 0 : Security [11] Session: network IPSEC[x.x.x.x-255.255.0.0] logged in from gateway [x.x.x.x] 16:24:28 tEvtLgMgr 0 : Security [12] Session: IPSEC[x.x.x.x]:12 physical addresses: remote x.x.x.x local x.x.x.x 16:24:28 tEvtLgMgr 0 : Security [12] Session: IPSEC[-]:13 physical addresses: remote x.x.x.x local x.x.x.x The highlighted text above is the name of the Branch Office Connection that was configured on the Contivity for the CP network. This name must match the one configured. NB: These Contivity messages do not necessarily indicate a successful connection, if the failure is on the CP side, you can still see these messages in the log. A successful connection is indicated by 1) both logs reporting successful, and 2) traffic being passed. Notes

1. Many documents suggest that the IKE and Rekey timeouts should be synchronised for this to work, however, in testing the connection worked even with different timeouts, although this may cause intermittent issues during the connection.

2. Many documents also say that ‘Supports Aggressive Mode’ is not supported, whereas in the test environment, it made no difference being on or off.