+ All Categories
Home > Documents > VPN Troubleshooting for Checkpoint

VPN Troubleshooting for Checkpoint

Date post: 22-Jan-2016
Category:
Upload: sachin-saini
View: 2,453 times
Download: 16 times
Share this document with a friend
Description:
vpn troubleshooting in checkpoint
27
VPN TROUBLESHOOTING : REFFER: http://www.cpug.org/forums/vpns-virtual-private-networks/4764- vpn-trouble-shooting.html Basics: IKE negotiation consists of two phases - Phase I (Main mode which is six packets) and Phase II (Quick Mode which is three packets). The $FWDIR/log/ike.elg file contains this information (once debugging is enabled). To enable debugging, you need to login to your firewall and enter the command "vpn debug on ; vpn debug ikeon" or "vpn debug trunc". Check Point have a tool called IKEView.exe which parses the information of ike.elg into a GUI making this easier to view. Note that another useful tool is "vpn debug on mon" which writes all of the IKE captured data into a file ikemonitor.snoop which you can open with wireshark or ethereal. PHASE1: negotiates encryption methods (DES/3DES/AES etc), the key length, the hash Algorithm (MD5/SHA1) and creates a key to protect the messages of the exchange. It does this in 5 stages: 1. Peers Authenticate using Certificates or a pre-shared secret. 2. Each peer generates a private Diffie-Hellman key from random bits and from that derives a DH public key. These are then exchanged. 3. Each peer generates a shared secret from its private key and its peers public key, this is the DH key. 4. The peers exchange DH Key material (random bits and mathematical data) and methods for PhaseII are agreed for encryption and integrity. 5. Each side generates a symmetric key (based upon the DH key and key material exchanged). In IkeView under the IP address of the peer, open the Main Mode Packet 1 - expand : > "P1 Main Mode ==>" for outgoing or "P1 Main Mode <==" for incoming >MM Packet 1 >Security Association >Prop1 PROTO_ISAKMP >Tran1 KEY_IKE You should then be able see the proposed Encryption Algorithm, Key Length, Hash Algorithm, Authentication Method, DH Group, and SA renegotiation params (life type - usually secs and duration). If your encryption fails in Main Mode Packet 1, then you need to check your VPN communities. Packet 2 ( MM Packet 2 in the trace ) is from the responder to agree on one encryption and hash algorithm Packets 3 and 4 arent usually used when troublshooting. They perform key exchanges and include a large number called a NONCE. The NONCE is a set of never before used random numbers sent to the other part, signed and returned to prove the parties identity. Packets 5 and 6 perform the authentication between the peers. The peers IP address shows in the ID field under MM packet 5. Packet 6 shows that the peer has agreed to the proposal and has authorised the host initiating the key exchange.
Transcript
Page 1: VPN Troubleshooting for Checkpoint

VPN TROUBLESHOOTING:

REFFER: http://www.cpug.org/forums/vpns-virtual-private-networks/4764-vpn-trouble-shooting.html

Basics:

IKE negotiation consists of two phases - Phase I (Main mode which is six packets) and

Phase II (Quick Mode which is three packets).

The $FWDIR/log/ike.elg file contains this information (once debugging is enabled).

To enable debugging, you need to login to your firewall and enter the command "vpn debug on ; vpn debug ikeon" or "vpn debug trunc".

Check Point have a tool called IKEView.exe which parses the information of ike.elg into a GUI making this easier to view.

Note that another useful tool is "vpn debug on mon" which writes all of the IKE captured data into a file ikemonitor.snoop which you can open with wireshark or ethereal.

PHASE1:

negotiates encryption methods (DES/3DES/AES etc), the key length, the hash Algorithm (MD5/SHA1) and

creates a key to protect the messages of the exchange.

It does this in 5 stages:

1. Peers Authenticate using Certificates or a pre-shared secret. 2. Each peer generates a private Diffie-Hellman key from random bits and from that derives a DH public key. These are then exchanged. 3. Each peer generates a shared secret from its private key and its peers public key, this is the DH key. 4. The peers exchange DH Key material (random bits and mathematical data) and methods for PhaseII are agreed for encryption and integrity. 5. Each side generates a symmetric key (based upon the DH key and key material exchanged).

In IkeView under the IP address of the peer, open the Main Mode Packet 1 - expand :> "P1 Main Mode ==>" for outgoing or "P1 Main Mode <==" for incoming

>MM Packet 1

>Security Association

>Prop1 PROTO_ISAKMP

>Tran1 KEY_IKE

You should then be able see the proposed Encryption Algorithm, Key Length, Hash Algorithm, Authentication Method, DH Group, and SA renegotiation params (life type - usually secs and duration).

If your encryption fails in Main Mode Packet 1, then you need to check your VPN communities.

Packet 2 ( MM Packet 2 in the trace ) is from the responder to agree on one encryption and hash algorithm

Packets 3 and 4 arent usually used when troublshooting. They perform key exchanges and include a large number called a NONCE. The NONCE is a set of never before used random numbers sent to the other part, signed and returned to prove the parties identity.

Packets 5 and 6 perform the authentication between the peers. The peers IP address shows in the ID field under MM packet 5. Packet 6 shows that the peer has agreed to the proposal and has authorised the host initiating the key exchange.

If your encryption fails in Main Mode Packet 5, then you need to check the authentication - Certificates or pre-shared secrets

Phase II –

Page 2: VPN Troubleshooting for Checkpoint

IPSec Security Associations (SAs) are negotiated, the shared secret key material used for the SA is determined and there is an additional DH exchange.

Phase II failures are generally due to a misconfigured VPN domain.

Phase II occurs in 3 stages:

1. Peers exchange key material and agree encryption and integrity methods for IPSec. 2. The DH key is combined with the key material to produce the symmetrical IPSec key. 3. Symmetric IPSec keys are generated.

In IkeView under the IP address of the peer, expand Quick Mode packet 1:> "P2 Quick Mode ==>" for outgoing or "P2 Quick Mode <==" for incoming

> QM Packet 1

> Security Association

> prop1 PROTO_IPSEC_ESP

> tran1 ESP_AES (for an AES encrypted tunnel)

You should be able to see the SA life Type, Duration, Authentication Alg, Encapsulation Mode and Key length.If your encryption fails here, it is one of the above Phase II settings that needs to be looked at.

There are two ID feilds in a QM packet. Under

> QM Packet 1

> ID

You should be able to see the initiators VPN Domain configuration including the type (ID_IPV4_ADDR_SUBNET) and data (ID Data field).

Under the second ID field you should be able to see the peers VPN Domain configuration.

Packet 2 from the responder agrees to its own subnet or host ID, encryption and hash algorithm.

Packet 3 completes the IKE negotiation.

If all of this works without any errors, then you may have previously initiated an invalid tunnel previously. You can use the VPN tunnel utility "vpn tu" to remove SA keys from the table.

A. If there is any additional information regarding two other frequent problems – one way only traffic and tunnel disconnections?

One way only traffic is generally the result of one peer not having correctly established a security association.

Most frequently this is due to the way in which Check Point combines adjacent IP address networks together into supernets. ie, if you have 192.168.0.0/24 and 192.168.1.0/24, Check Point will supernet this into 192.168.0.0/23.

This is done to reduce the number of keys required and hence reduce the load on the VPN gateway. However, other VPN devices do not follow this methodology, so depending upon the version of VPN-1 you are using, you may need to set IKE_use_largest_possible_subnets or correctly configure the VPN communities tunnel management (one vpn per pair of hosts, per subnet pair or per gateway). See SecureKnowledgebase article Solution ID: #sk26336.

Tunnel disconnections can be caused either a physical connectivity problem or routing problems or once again, a mismatch in the VPN security associations. Be particularly careful with VPNs to Cisco in this regards. Plenty of times I've seen people confused between seconds and minutes! I've also seen that sometimes the Cisco ends of VPNs dont want to reset the SAs when told to by the Check Point end.

B. If there is any information regarding ike.elg + vpn.elg explanation. I familiar with the ike/ipsec processes but those 2 files are still no easy to understand (I know there is a tool called ikeview but I don’t work for organization considered as CSP so it’s seems like I’ll never put my hands on this tool) ?

From my answer above, you can see that my statement about "Most VPN debugging consists of looking at the IKE negotiation" to be true. And it is most unlikely that you will need to look into the vpnd.elg file. With respect to obtaining ikeview.exe.

IMPORTANT PATHS:

1.Where are database revision control files stored?

$FWDIR/conf/db_versions/repository/

Cisco ASA order of operations

1. FLOW-LOOKUP- This will check for existing connections. I a connection exists, the flow is automatically allowed

2. ROUTE-LOOKUP - This is the inbound route lookup which includes reverse patch, if enabled.

3. Inbound ACCESS-LIST- Checks for an interface ACL

4. CONN-SETTINGS - Application layer checks (Class maps)

5. IP-OPTIONS- RFC 791

Page 3: VPN Troubleshooting for Checkpoint

6. NAT

7. Outbound ACCESS-LIST (if an outbound access list exists on the egress interface).

9.FLOW-CREATION

10.ROUTE LOOKUP - Destination route lookup

How to immediately know if you are logged into the active or standby firewal on ASA

The prompt (introduced in 7.2(1)) command allows you to customize the hostname of the ASA to include dynamic elements.

prompt state will display the state of the firewall.for example:lab-dev-01# config tlab-dev-01 (config)# prompt state  lab-dev-01/act(config)#

•act—Failover is enabled, and the unit is actively passing traffic.

•stby— Failover is enabled, and the unit is not passing traffic and is in a standby, failed, or other non-active state.

•actNoFailover—Failover is not enabled, and the unit is actively passing traffic.

•stbyNoFailover—Failover is not enabled, and the unit is not passing traffic. This might happen when there is an interface failure above the threshold on the standby unit.

How to determine the path and interface of a host on SPLAT

[Expert@lab1]# ip route get 192.168.1.1192.168.1.10 via 192.168.19.38 dev eth2  src 192.168.19.1     cache  mtu 1500 advmss 1460

How to ping the inside interface of an ASA through a VPN tunnel

This is typically used for testing VPNS.

#conf t(config)# management-access inside

Step backwards for VPN supernetting in R71

It was recently brought to my attention that Checkpoint's infamous VPN supernetting in R71 can no longer be fixed by changing the VPN Advanced Tunnel Options to "1 VPN Per Pair of Hosts".

As with R55, you have to change the following:$FWDIR/conf/Objects_5_0.C file. Change “Support Subnets for Key Exchange” to “false”. 

It is also believed that migrated VPNS that were previously configured using the Advanced Tunnel Options retain their settings, but new VPNs will not work. If anyone has any more information on this, I would appreciate the input.

It is believed that R75 has gone back to the use of the Advanced VPN Tunnel Options setting.

Cluster cannot be empty" or "only one interface defined" error on Checkpoint R70 and R71

Checkpoint has confirmed that this is a bug that occurs occasionally when pushing policy to clusters.  Example:

Firewall and Address Translation Policy Verification: Verifier warnings: There is only one interface defined for object . At least one more interface must be configured for this object in order to use the Anti-Spoofing feature.

  or  

"Verifier warnings: A Cluster cannot be empty. It must have Cluster members"

To resolve the issue, open the cluster object for the gateway that you are pushing policy for,  and go into the Topology. Click OK, then OK on the Cluster object and Save. The error should go away.

Page 4: VPN Troubleshooting for Checkpoint

Checkpoint R71 bug that will cause migrations to fail

Checkpoint has acknowledged that there is a bug in R71 that will cause any policy migrations from older versions to fail if there is a tilde (~) in the name of a policy being migrated. This is true even if the policy is not used.Therefore any policies with a tilde in the name should be renamed before migrating.

ERROR: This license does not allow configuring more than 2 interfaces with nameif and without a "no forward" command on this interface or on 1 interface(s) with nameif already configured.

This error will occur during the configuration of a ne wVLAN on an ASA 5505.

This error occurs because there is only a Base license installed on the ASA. The license will need to be upgraded to a Security Plus license.

A base license will only allow 3 VLANS to be created.

Everything you need to know about troubleshooting VRRP on Nokia Checkpoints

VRRP failover happens when one of the following events takes place:-a monitored interface looses its link state-VRRP hello packets from the master not seen on the secondary device-a critical Checkpoint service or daemon fails to report its status. This requires FW Monitoring to be turned on in Voyager. If turned on, whenever the clock is set backwards, a failover will also occur.

tcpdump -nni eth1 proto VRRP

The packets will contain the vrid and priority.When a failure occurs, the failed device sends out a priority 0 message on all good interfaces. This tells the secondary to take over.

Example:PrimaryHA-fw1[admin]# tcpdump -i eth-s1p1c0 proto vrrptcpdump: listening on eth-s4p2c000:46:11.379961 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 100 pri 100 [tos 0xc0]00:46:12.399982 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 100 pri 100 [tos 0xc0]00:46:13.479985 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 100 pri 100 [tos 0xc0]00:46:14.560007 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 100 pri 0 [tos 0xc0]

If both firewalls are broadcasting vrrp, and the packets are not seen by the other firewall, there could be a communication problem between the firewalls.

Also ensure that the vrid matches on both firewalls.

Proper VRRP failovers usually only cause 1 or 2 packets lost .

VRRP multicast address is 224.0.0.18

To capture vrrp traffic in fw monitor:

fw monitor -e “accept ip_p = 112;”

Clishshow vrrpThis will show you which devices are in master and backup

Example:PrimaryFW-A> sh vrrpVRRP StateFlags: On6 interface enabled6 virtual routers configured0 in Init state0 in Backup state6 in Master statePrimaryFW-A>PrimaryFW-A> exitBye.PrimaryFW-A[admin]# SecondaryFW-B[admin]# iclidSecondaryFW-B> sh vrrpVRRP StateFlags: On6 interface enabled6 virtual routers configured0 in Init state4 in Backup state

Page 5: VPN Troubleshooting for Checkpoint

2 in Master stateSecondaryFW-B>SecondaryFW-B> exit

show vrrp interfacesDetailed configuration of VRRP, including priority, hello interval, and VRID

clish -c "show interfacemonitor"Displays interface transitions

cphaprob -i listDisplays Checkpoint critical processes and their timeouts.

To log critical process failures:ipsctl -w net:log:partner:status:debug 1

That will log to the console and to /var/log/messages. If you want to turn off: ipsctl -w net:log:sink:console 0

To change the timeout value of a monitored process:cphaprob -d [device] -t [timeout] -s [state] -p register

Resolving local logging issues on Checkpoint

If logs are not appearing in Smartview Tracker, they are probably logging locally.To determine if logs are being stored locally on the gateway, go to $FWDIR/log.Locate the fw.log file and see if it's size is incrementing. There may also be additional fw*.log files that have rolled over.To resolve the issue, first try restarting the MLM (in a Provider environment or the Log Services in a Smartcenter Server environment).Next, restart the firewall services on the gateway (fw kill fwd followed by fwd).If that does not work, try restarting the firewall.

Once resolved, you can pull the stored logs from the gateway by running "fw fetchlog " from the log server. In R70, there is also an option to fetch logs in Smartview Tracker (Tools>Remote Files Mgmt)

Configuring SNMP on SPLAT

step 1: service snmpd restartstep 2: edit /etc/snmp/snmpd.users.conf and replace public with your actualsnmp community stringstep 3: service snmpd restartstep 4: netstat -an | grep 161

for checkpoint snmpd port 260:

step 1: modify the $FWDIR/conf/snmp.C file and place the actual snmpcommunity inside the read and write (). If you leave the write empty,it will use "private" as the community string. This is a security risk.

step 2: run sysconfig and start the checkpoint snmpd extension

step 3: perform cpstop;cpstart

step 4: netstat -an | grep 260

Creating a Read Only SPLAT user

Creating a user1. SSH to the firewall where account will be setup on.2. From the command line type “adduser ”, here we will add the user with username testuser. The command should read “adduser testuser”3. Input the desired password when prompted to do so

Changing the users shell1. Open the passwd file for editing by typing “vi /etc/passwd”2. Find the line corresponding to the user you just created. If you have created a user with username “testuser”, the line you are looking for is “testuser:x:0:0::/home/test:/bin/cpshell”3. Change the users shell, to do this we will change “/bin/cpshell” to “/path/to/shell”.Before the change the line should read:“testuser:x:0:0::/home/test:/bin/cpshell”After the change the line will read:“testuser:x:0:0::/home/test:/etc/scripts/myshell.sh”

Page 6: VPN Troubleshooting for Checkpoint

Checkpoint port list

TCP 256: CA and DH key exchange, net topology fetch on older SC versions, and port used to push policy to remote firewalls.TCP 257: LoggingTCP 258: Mgt console listens for remote GUI connectionsTCP 259: Client Auth via telnetUDP 259: manages encrypted sessionsUDP 260: SNMP for the Checkpoint daemonTCP 262: Single Sign-on daemon. TCP 264: SC topology fetch.UDP 500: ISAKMPTCP 900: HTTP client auth.TCP 4532: Session Auth agentTCP 18181: Content Vectoring ProtocolTCP 18182: URL Filtering Protocol.TCP 18183: Suspecious Activity Monitoring for IPS.TCP 18186: SIC between OPSEC products and the gateway.TCP 18190: Gateway listens for management clients.TCP 18191: CPD process for communications such as policy installation and certificate revocation. TCP 18192: CPD monitoring

Cisco ASA Policy Based Nat

Example:Source address 10.1.1.1 should be translated to 192.168.1.1 when going to 172.16.1.1 and translated to 192.168.1.2 when going to 172.16.1.2

access-list policy_nat1 permit ip host 10.1.1.1 host 192.168.1.1access-list policy_nat2 permit ip host 10.1.1.1 host 192.168.1.2

static (inside,outside) 172.16.1.1 access-list policy_nat1static (inside,outside) 172.16.1.2 access-list policy_nat2

Cisco ASA Reverse Path Forwarding

Reverse Path Forwarding

RPF errors are typically NAT related (traffic is natted one way in one direction and another way in the other direction).

Example: --->no nat<--- hide nat

Example of this error:

 %ASA-5-305013: Asymmetric NAT rules matched for forward and reverse flows; Connection for icmp src outside:192.168.25.100 dst dmz:192.168.28.12 (type 8, code 0) denied due to NAT reverse path failure

The other reason is if RPF checking is turned on and the source host comes in on an interface where a route is not defined for the host. This type of RPF check must be configured on a per interface basis, which will cause the firewall to examine the source IP of each packet. This also adds a little additional overhead.

To turn on interface RPF checking run the following interface config command:ip verify reverse-path interface outside

Great Cisco TAC podcast on Anyconnect

This podcast covers an overview of Anyconnect as well as some great troubleshooting procedures.http://cisco-podcast.streamguys.net/cdc/security/tac/TACSecurityShow_episode_11.mp3

Troubleshooting VRRP

Check the status of the interfaces

In this example, both firewalls believe that they are in a master state.

FW-1[admin]# iclidFW-1> sh vrrp

VRRP State

Page 7: VPN Troubleshooting for Checkpoint

Flags: On6 interface enabled6 virtual routers configured0 in Init state0 in Backup state6 in Master stateFW-1>FW-1> exit

FW-2[admin]# iclidFW-2> sh vrrp

VRRP StateFlags: On6 interface enabled6 virtual routers configured0 in Init state4 in Backup state2 in Master state

A TCPDUMP can confirm that VRRP packets are reaching each interface.

On the Primary:

FW-1[admin]# tcpdump -i eth-s4p2c0 proto vrrptcpdump: listening on eth-s4p2c000:46:11.374424 O 192.168.10.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]00:46:12.344334 O 192.168.10.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]

Secondary:

FW-1[admin]# tcpdump -i eth-s4p2c0 proto vrrptcpdump: listening on eth-s4p2c000:19:38.533454 O 192.168.10.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]00:19:39.544322 O 192.168.10.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]Now you can see that the interface on both the primary and the secondary firewalls are broadcasting vrrp multicasts. This is because the vrrp multicasts are not reaching the firewalls interfaces. This means there is a communication breakdown which can be possibly caused by network issues.

In another example you will see that the VRIDS dont match

FW-1[admin]# tcpdump -i eth-s4p2c0 proto vrrp

00:46:11.206994 I 10.10.10.1 > 224.0.0.18: VRRPv2-adver 20: vrid 103 pri 95 [tos 0xc0]00:46:11.379961 O 192.168.1.1 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 100 [tos 0xc0]

FW-1[admin]# tcpdump -i eth-s4p2c0 proto vrrp00:19:38.507294 O 192.168.1.2 > 224.0.0.18: VRRPv2-adver 20: vrid 102 pri 95 [tos 0xc0]00:19:38.630075 I 10.10.10.2 > 224.0.0.18: VRRPv2-adver 20: vrid 103 pri 100 [tos 0xc0]

Usefull Nokia IPSO Commands

newimage -R -k -l ipso.tgz - install a new IPSO image

newpkg –i installs software from given location (firewall software, VPN accel driver, etc)

voyager –e 0 80 resets voyager after a failed ssl config attempt

dbpasswd admin -Changes the password from the command line

ipsofwd on admin -turns on ip forwarding when firewall is stopped

ipsofwd list -displays ipso properties (flowpath, etc)

ipsofwd slowpath -turns off flows (flowpath turns back on)

iclid -vrrp utility that shows status

- show vrrp -iclid command that shows # of interfaces and their respective states

- get vrrp -shows iclid stats: active interfaces/checksum/version/id

-show vrrp interface -displays interface stats for VRRPboot –s {from > prompt at boot time) boots into single-user mode

Nokia IPSO has 2 shells, IPSO and Clish.

After logging in, you are in the IPSO shell. To enter the Clish shell, type "clish"

To remove old config:Either rm /active/config or config/active depending on version.

Page 8: VPN Troubleshooting for Checkpoint

Usefull Checkpoint Commands

o view the active connections table: fw tab -t host_table –s

To pull the latest policy from the management station: fw fetch

Display the name of the policy installed and the date it was received: fw stat

View the Checkpoint version installed: fw ver

Display cpu, memory, and disk usage: fw ctl pstat

Delete all hosts from the connections table: fw tab -t host_ip_addrs –x

Display logs on the firewall for a specific IP: fw log –n –ft | grep

Troubleshoot source/destination access issues: fw monitor -m iIOo -e 'accept src=10.33.76.82 and dst=10.33.76.82;'

Manage VPN connections (view and delete): vpn tu

Turn on debugging for VPN's: vpndebug on and vpn debug ikeon

This will create 2 files in $FWDIR/logs. vpnd.elg (this can be viewed on the firewall using cat. It will show highlevel VPN connection information), and ike.elg (this is the bread and butter of Checkpoint VPN troubleshooting. Click here to read my ikeview guide).

Display SIC key: cp_conf sic get

High Availabiliy: cphaprob stat -display HA status

cphaprob -i -display HA interface stats

cphastop/cphastart -stop/start HA

View license key installed: cplic print

Delete all active hosts: fw tab -t host_ip_addrs –x

Common Clish commands on Nokia IPSO appliances

---setting default gatewayset static-route default nexthop gateway address 192.168.29.2 priority 1 on

---adding static routesset static-route 172.23.124.150/32 nexthop gateway address 192.168.29.50 on

---Add proxy arpadd arpproxy address 192.168.29.56 macaddress 0:a0:8e:7d:13:d0add arpproxy address 192.168.29.57 macaddress 0:a0:8e:7d:13:d0

---Add an interfaceset interface eth1 speed 100M duplex full active onadd interface eth1c0 address 192.168.29.54/24set interface eth1c0 enable

---VRRP

set vrrp accept-connections onset vrrp coldstart-delay 60

set vrrp interface eth1c0 monitored-circuit vrid 54 monitored-interface eth2c0 onset vrrp interface eth1c0 monitored-circuit vrid 54 monitored-interface eth2c0 priority-delta 10set vrrp interface eth1c0 monitored-circuit vrid 54 monitored-interface eth3c0 onset vrrp interface eth1c0 monitored-circuit vrid 54 monitored-interface eth3c0 priority-delta 10set vrrp interface eth1c0 monitored-circuit vrid 54 priority 100set vrrp interface eth1c0 monitored-circuit vrid 54 hello-interval 1set vrrp interface eth1c0 monitored-circuit vrid 54 vmac-mode default-vmacset vrrp interface eth1c0 monitored-circuit vrid 54 backup-address 192.168.29.1 on

---Set ntp servers

add ntp server 10.1.1.2 version 3 prefer yesadd ntp server 10.1.1.1 version 3 prefer yes

---Setting Time zone

set date timezone-city "Greenwich (GMT)"

---Add hostname

set hostname testbox

---Add Host address assignments

add host name testbox ipv4 192.168.29.54

Page 9: VPN Troubleshooting for Checkpoint

Nokia IPSO Password reset

Boot the Nokia device into single user mode

To boot an IP440 into single user mode first restart the box.. When you see the "boot:" prompt enter "-s" and press "enter" within 10 seconds. When it boots into single user mode it will ask for the shell, just press "enter" to accept the default "sh."

To boot an IP500 or higher into single user mode, first restart the box. When you will see the prompt "Entering autoboot mode. Type any character to enter command mode." You have 5 seconds to press any key.

To boot at IP300 device into single user mode, first restart the box. When you see the prompt "Verifying DMI Pool Data" press the number 1. Then you will see a "Type any character to enter command mode." You now have 5 seconds to press any key. After pressing any key type "boot -s" to enter single user mode.

Change Password in IPSO 3.5 and Higher

Run "/etc/overpw" from the single user shell and follow the prompts to change the password. Type "reboot" to boot into multi-user mode, go into voyager and change to a permanent password.

Cisco VPN Troubleshooting Guide

Cisco PIX 7.0 VPN Troubleshooting

Quick overview of IPSECIt is important to understand how IPSEC works in order to understand how to troubleshoot a VPN connection. This is a quick overview of IPSEC and is by no means a complete detailed guide.

IPSEC is a suite of protocols, defined in RFC 2401, that is used to protect information as it travels from one private network to another private network over a public network.

IPSEC consists of Security Protocols (AH and ESP), Key Management (ISAKMP, IKE, and SKEME), and Algorithms (3DES, AES256, etc).

ISAKMP defines the procedures and packet formats used to establish, negotiate, and modify Security Associations. ISAKMP communicates over UDP 500.

Security Protocols consist of AH (Authentication Header) and ESP (Encapsulating Security Payload). AH communicates over IP 51 and provides data authentication, integrity, and replay protection (for man in the middle attacks), but does not provide confidentiality. It is important to understand that AH encapsulates the IP packet but does not encrypt it.ESP communicates over IP 50 and provides the same service as AH in addition to providing data confidentiality by encrypting the original payload and encapsulating the packet.

SA’s (Security Associations):In order to have an IPSEC conversation, you first need a security association. Each device must agree on the policies or rules of the conversation by negotiating these policies with their potential peers. The SA represents a unidirectional instance of a security policy for a given connection.

Main mode IPSEC packet exchange:--Initiator--- ---Responder-------------pk#1—Policy Proposal------><-------pk#2---Policy Accept/Reject-- ----------pk#3---DH Exchange--------><-------pk#4---DH Exchange---------- ----------pk#5---ID/Hash-------------><------pk#6---ID/Hash--------------->

Packet handling order:

Step 1 Access lists applied to an interface and crypto map are used by Cisco IOS software to select interesting traffic to be encrypted.Step 2 Cisco IOS software checks to see if IPSec SAs have been established.Step 3 If the SA has already been established by manual configuration using the crypto ipsec transform-set and crypto map commands or has been previously set up by IKE, the packet is encrypted based on the policy specified in the crypto map and is transmitted out of the interface.Step 4 If the SA has not been established, Cisco IOS software checks to see if an IKE SA has been configured and set up.Step 5 If the IKE SA has been set up, the IKE SA governs negotiation of the IPSec SA as specified in the IKE policy configured by the crypto isakmp policy command, the packet is encrypted by IPSec, and it is transmitted.Step 6 If the IKE SA has not been set up, Cisco IOS software checks to see if certification authority (CA) has been configured to establish an IKE policy.Step 7 If CA authentication is configured with the various crypto ca commands, the router uses public and private keys previously configured, obtains the CA's public certificate, gets a certificate for its own public key, and then uses the key to negotiate an IKE SA, which in turn is used to establish an IPSec SA to encrypt and transmit the packet.

Configuring Phase 1:The first 2 octets of IPs have been replaced with "y.y."Phase I is not configured on a per connection basis. When a Phase I connection is being established, configured ISAKMP policies will be tried one at a time until a match is found.

Example of an ISAKMP policy:#isakmp policy 20 authentication pre-share#isakmp policy 20 encryption 3des#isakmp policy 20 hash md5#isakmp policy 20 group 2#isakmp policy 20 lifetime 43200

Troubleshooting Phase I:

Check the syslogs

Show run isakmp This will show the isakmp policies for all VPN connections. To view a specific ISAKMP policy type show run isakmp | grep

Page 10: VPN Troubleshooting for Checkpoint

show vpn-sessiondb detail l2l

Show crypto isakmp sa detail – This command will display the state of Phase I of the IPSEC tunnel. A state of MM_Active indicates that Phase I was successfully completed. If Phase I does not complete, refer to the table below to find out exactly what state the Phase I connection is currently in. This will give you an indication of where the problem has occurred. More specific information can be found by running a debug(discussed later).

State DescriptionOAK_MM_No_STATE This is the initial state of Phase I. If you see Phase IIn this state for longer than a few seconds, this is anindication that a failure of tunnel establishment forPhase I has occurred.

OAK_MM_SA_SETUP The peers have agreed on parameters for the ISAKMPSA. Phase I will be in this state after packet 1 and packet 2 exchange of the Main Mode negotiation (see above).

MM_WAIT_MSG The firewall is waiting on the remote end device to respond with DH and public key.

OAK_MM_KEY_EXCH The peers have exchanged DH public keys and have generated a shared secret.

OAK_MM_KEY_AUTH The ISAKMP SA has been authenticated.

The debug crypto isakmp 5 command will display real time information on every step of the Phase I connection. Debug level 5 should be sufficient for most troubleshooting however level 7 provides more detailed information if necessary.Please note that you cannot limit the debug output to a specific tunnel.

IKMP_NO_ERROR_NO_TRANS indicates a matching transform set was not found

No Proposal Chosen=isakmp policy mismatch

syslog sample of a completed connection:Mar 10 2008 18:47:05: %PIX-3-713119: Group = y.y.41.250, IP = y.y..41.250, PHASE 1 COMPLETED

Sample Debug output:The following shows the initiation of the first packet for an IPSEC tunnel.58534 02/27/2004 07:42:38.600 SEV=4 IKE/41 RPT=8619 y.y.11.49IKE Initiator: New Phase 1, Intf 2, IKE Peer

The following indicates that the IKE Phase I policy was accepted by the remote gateway.58534 02/27/2004 07:42:38.600 IP = y.y.11.49, Oakley proposal is acceptable

This indicates Phase I has completed.58534 02/27/2004 07:42:38.600 Group= y.y.11.49, IP=y.y.11.49, Oakley begin quick mode

The following indicates that the remote gateway has indicated that none of the policies are acceptable.5|Oct 02 2006 09:41:41|713904: IP = y.y.138.12, Received an un-encrypted NO_PROPOSAL_CHOSEN notify message, dropping

To clear the Security Associations related to Phase 1, use the clear crypto isakmp command. This will clear ALL of the SA’s currently built on this firewall.

To confirm that the IPSEC packets are reaching the firewall, a capture can be created for all UDP 500 traffic.First create an access-list for the traffic you would like to capture.Access-list capture1 permit udp any any eq 500

Next create a capture.Capture cap1 access-list capture1 interface outside

Next display the results of the capture.Show capture cap1 detail

ciscoasa#show capture cap1 detail1: 13:04:06.284897 192.168.1.50 > 192.168.1.1: UDP:500

View capture on webhttps://capture/pcap/cap1

View pre-shared keys:more system:running-config

Configuring Phase 2:A transform set combines encryption method and authentication method. During the IPSec security association negotiation with ISAKMP, the peers agree to use a particular transform set to protect a particular data flow. The transform set must be the same for both peers.You can create multiple transform sets, and then specify one or more of these transform sets in a crypto map entry.You can view previously created transform sets by typing the show crypto ipsec transform-set command. If the desired transform set has not been previously defined, the crypto ipsec transform-set command is used to create it.

Example:#(config)crypto ipsec transform-set 3desmd5 esp-3des esp-md5-hmac

An access-list is used to define the “interesting traffic” or the traffic that should be encrypted and allowed through the VPN Tunnel. The access-list should always be defined from local to remote. The subnet sizes need to match on the remote gateway.

Example:#(config) access-list tunnel1 extended permit ip y.y..191.0 255.255.255.0 host y.y..155.12

If port filtering is being used, and traffic is being initiated from the remote side, the destination port of the remote host must be the source port of the local matching acl.

Page 11: VPN Troubleshooting for Checkpoint

A tunnel group is used to identify specific connection parameters and the definition of a group policy. The default tunnel groups are DefaultRAGroup (used for Remote Access tunnels) and DefaultL2Lgroup(used for IPSEc Lan-to-Lan tunnels).

Example:#(config)tunnel-group y.y.155.1 type IPsec_l2l#(config)tunnel-group y.y.155.1 ipsec-attributes#(config-attributes) pre-shared-key abc123

The crypto map ties together several components that define the VPN tunnel. This is where the peer defined in the tunnel-group command is tied to the access-list and transform-set. The crypto map must be assigned a unique map id #. To view the previously used crypto map id numbers run the show ru crypto command.

Example:#(config)crypto map mymap 10 match address tunnel1#(config)crypto map mymap 10 set peer y.y,155.1#(config)crypto map mymap 10 set transform-set 3desmd5

Nat considerations:If a local address is going to be natted outbound, the crypto acl should use the outside ip address.

Troubleshooting Phase II:Check syslogs

Show crypto ipsec sa- This command shows the output of the IPSEC SA’s. The SA will include the ip address of the local and remote endpoints, encryption domains (interesting traffic), transform set (what encryption and hash is being used), key lifetime, and # of packet encrypt/decrypts.

debug crypto engine—Displays the traffic that is encrypted.

Example of an IPSEC SA:This shows the crypto map used for this connection.Crypto map tag: vpn_map, seq num: 130, local addr: x.x.160.45

The following line shows the crypto acl that includes the traffic to be protected.access-list VPN-CIDS704976 permit ip x.x.190.0 255.255.254.0 host 10.2 5.4.80local ident (addr/mask/prot/port): (x.x.190.0/255.255.254.0/0/0)remote ident (addr/mask/prot/port): (10.25.4.80/255.255.255.255/0/0)current_peer: y.y.227.136

Encrypts indicate that this side is encrypting and sending traffic. Decrypts indicates that the other side is sending traffic.#pkts encaps: 5, #pkts encrypt: 5, #pkts digest: 5#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0#pkts compressed: 0, #pkts decompressed: 0#pkts not compressed: 5, #pkts comp failed: 0, #pkts decomp failed: 0#send errors: 0, #recv errors: 0

This lists the local and remote endpoints.local crypto endpt.: x.x.160.45, remote crypto endpt.: y.y.227.136

path mtu 1500, ipsec overhead 74, media mtu 1500current outbound spi: 2AFEA5C7

There is a separate sa for inbound and outbound.inbound esp sas:spi: 0x9D111D2A (2635144490)transform: esp-aes-256 esp-sha-hmac nonein use settings ={L2L, Tunnel, PFS Group 5, }slot: 0, conn_id: 317225, crypto-map: vpn_mapsa timing: remaining key lifetime (kB/sec): (4275000/28789)IV size: 16 bytesreplay detection support: Youtbound esp sas:spi: 0x2AFEA5C7 (721331655)transform: esp-aes-256 esp-sha-hmac nonein use settings ={L2L, Tunnel, PFS Group 5, }slot: 0, conn_id: 317225, crypto-map: vpn_mapsa timing: remaining key lifetime (kB/sec): (4274999/28789)IV size: 16 bytesreplay detection support: Y

Clear crypto ipsec sa peer will clear the Phase 2 SA’s for a given peer.

debug crypto ipsec—Displays the IPSec negotiations of phase 2.

No Valid SA/ Identity mismatch – Transform set or crypto acl

Sample Debug output:The following shows that the tunnel group configuration was found.Oct 26 15:42:43 [IKEv1]: IP =y.y.205.92, Connection landed on tunnel_group y.y,.205.92

Sample syslog errors:

This shows interesting traffic ACL getting exchanged.1754 11/29/2001 16:20:18.500 SEV=7 IKEDBG/0 RPT=546 y.y.205.92Transmitting Proxy Id:Remote host: 192.168.1.1 Protocol 0 Port 0Local host: 10.64.10.9 Protocol 0 Port 0

Completion of Phase II.1949 11/29/2001 16:20:18.540 SEV=4 IKE/49 RPT=3 y.y.205.92Security negotiation complete

Page 12: VPN Troubleshooting for Checkpoint

Responder, Inbound SPI = 0x11a56495, Outbound SPI = 0xb17718a5

Mar 10 2008 18:47:05: %PIX-5-713120: Group = y.y.41.250, IP = y.y.41.250, PHASE 2 COMPLETED (msgid=0f78e513)

Pre-shared key mismatch.1754 11/29/2001 16:20:18.500 Group = 172.16.172.63, IP = 172.16.172.63, Received an un-encrypted PAYLOAD_MALFORMED notify message, dropping.

Pre-shared key mismatch reported by the report peer(receiving peer)::713903: Group = 172.1.12.1, IP = 172.1.12.1 ERROR. peer has indicated that something is wrong with our message. This could indicate a pre-shared key mismatch.

Transform-set mismatch.1754 11/29/2001 16:20:18.500 Group = 172.16.172.63, IP = 172.16.172.63, Received non-routine Notify message: No Proposal Chosen

Transform-set mismatch on remote peer(receiving peer):713904” IP = 10.51.16.1, Received encrypted packet with no matching SA, dropping713048: IP = 10.51.16.1 Error processing payload. Payload ID 1

The following indicates that the remote gateway is not finding matching interesting traffic.1754 11/29/2001 16:20:18.500 Group = y.y.172.63, IP = y.y.172.63, Received non-routing Notify message: Invalid ID info (18)

The following indicates that the local gateway is not finding matching interesting traffic.1754 11/29/2001 16:20:18.500 Group =y.y.172.63, IP = y.y.172.63, Static Crypto Map check, map = mymap, seq = 10, ACL does not match proxy IDs src:192.168.1.0 dst:192.168.2.0

PFS mismatch:

713068: Group – 172.1.12.1, Received non-rouing Notify message; No Proposal chosen (14)PFS turned on on the remote peer. Local peer reports the following:713902; Group = 10.51.16.1. QM FSM error (p2 struct &0x296fde8, mess id 0x518e80d)!QM FSM is a generic message indicating that the phase II connection was rejected by the remote peer.

This indicated that the remote peer is natting:%ASA-4-402116: IPSEC: Received an ESP packet (SPI= 0x72DEC2AA, sequence number= 0x41) from y.y.28.178 (user= y.y.28.178) to y.y.83.194. The decapsulated inner packet doesn't match the negotiated policy in the SA. The packet specifies its destination as y.y.83.194, its source as y.y.28.178, and its protocol as 1. The SA specifies its local proxy as y.y.10.16/255.255.255.240/0/0 and its remote_proxy as y.y.63.0/255.255.255.0/0/0.

When reverse route is turned on:

Jan 26 2009 18:15:07: %ASA-6-713211: Group =y.y43.160, IP = y.y.43.160, Adding static route for L2L peer coming in on a dynamic map. address: 192.168.8.5, mask: 255.255.255.255Jan 26 2009 18:57:54: %ASA-6-713213: Group = y.y.43.160, IP =y.y43.160, Deleting static route for L2L peer that came in on a dynamic map. address: 192.168.8.5, mask: 255.255.255.255

Saturday, February 28, 2009

Checkpoint Tables and the FW Tab Command

The fw tab command displays the contents of the INSPECT tables. These tables hold all state information on the firewall, including connections, VPNS, nats, etc.

The following options are commonly used:-s provides a summary the tables-t tname only displays the requested table-x tname delete all entries in the specified table-d debug mode-all all tables

Useful tables:

host_table

This host_table holds the IP addresses of internal machines protected by the VPN-1/FireWall-1 NG Enforcement Module. The table only exists where the VPN-1/FireWall-1 license is limited. The maximum number of entries in this table is the licensed number of internal machines.

arp_table

The arp_table holds the IP addresses for which the machine is willing to proxy ARP. Proxy ARP is sometimes required when using NAT. If IP addresses that the machine is to resolve are specified in local.arp , this table can be used to check that the IP addresses were correctly specified. The automatic ARP feature also uses this table to cause the machine to resolve translated IP addresses.

Entry format:

<!--[if !supportLists]-->1. <!--[endif]-->IP address (name resolving may occur).

<!--[if !supportLists]-->2. <!--[endif]-->MAC address of gateway machine interface that will answer ARP requests for the IP address.

Page 13: VPN Troubleshooting for Checkpoint

<!--[if !supportLists]-->3. <!--[endif]-->Interface name (optional).

<!--[if !supportLists]-->4. <!--[endif]-->Name of the gateway interface that will proxy ARP for IP addresses.

The fwx_alloc table uses the following formats.

First entry: < 0, hiding IP address, IP protocol, first high port used; next high port to be allocated>

The first field is a space holder and is always 0. The first high port to be used is always 10000.

fwx_alloc

The fwx_alloc table holds information about the allocation of ports for the translated packets.

fwx_cntl_dyn_tab

The fwx_cntl_dyn_tab table holds information about the allocated IP addresses from the IP Pool of the Enforcement Module, and the connections using the IP addresses.

EXAMPLEattributes: keep, expires never, limit 25000, hashsize 512, free function 40550248 0 <0a010104,> 00000001, 00000e10, 000000c3>

fwx_auth

The fwx_auth table holds the original information of a folded connection, so that the Security Server will know the original destination IP and port of the connection.

EXAMPLEattributes: expires 300, limit 25000, refresh, keep c7cb47e3, 00000017; 286/300>

sam_blocked_ips

All IP and network addresses that were stipulated in SAM requests, are shown in sam_blocked_ips , with a requests counter for each prototype of filter that is enforced over each certain IP and network address. Note the overshadowed requests are also accounted for, if present.

EXAMPLE-------- sam_blocked_ips --------dynamic, id 8141, attributes: keep, limit 25000, hashsize 512 <05050505;> 00000000, 00000000, 00000000>

host_ip_addrs

The host_ip_addrs table contains the list of IP addresses in the VPN-1/FireWall-1 NG Enforcement Module.

fwx_ip_lookup_tab

The fwx_ip_lookup_tab table holds information used for IP Pool allocation queries.

EXAMPLEattributes: keep, limit 25000, hashsize 512

fwz_crypt_pending

The fwz_crypt_pending table is used to record a possibly encrypted connection that should obtain their encryption/decryption key. This table is accessed from the VPN kernel, and the VPN daemon. The kernel can obtain a new key from this table stored by the daemon, which negotiated the key exchange with a trusted peer. This table also passes error messages. The fwz_crypt_pending table is dynamic.

Page 14: VPN Troubleshooting for Checkpoint

forbidden_tab

Each embedded VPN-1/FireWall-1 system has a feature that indicates how many hosts can be located ‘behind’ it. The number of hosts can be set to unlimited. This limitation is enforced in the Inspect code using the macro COUNT_HOST .

COUNT_HOST records each packet that comes from the internal interface in a table until the limit is exceeded

IKE_SA_table

IKE SAs are stored in IKE_SA_table . The table entries have four possible formats:

<!--[if !supportLists]--> <!--[endif]-->The top two formats are only used on SecuRemote.

<!--[if !supportLists]--> <!--[endif]-->All non-expired SAs are stored using format 2.

<!--[if !supportLists]--> <!--[endif]-->Format 1 is used to store and retrieve the latest IKE SA.

<!--[if !supportLists]--> <!--[endif]-->Table entries are used to conduct IKE Quick Mode negotiation of IPSEC_SA .

<!--[if !supportLists]--> <!--[endif]-->Entries are extracted from this table when the vpn daemon is trapped for IPSEC_SA renewal. IKE daemon tries to use previously negotiated IKE SA.

<!--[if !supportLists]--> <!--[endif]-->The IKE_SA_table is dynamic.

ATTRIBUTES

expires 3600

limit 25000

sync

keep

hashsize 512

kbuf 1

KEYS

PeerAddress ipaddr IKE peer address.

Me ipaddr on SecuRemote it is the IP address used by SecuRemote when it negotiated this IKE SA; field is used to prevent using an old IKE SA negotiated before SecuRemote obtained a new IP address; if initiator or responder

CookieI u_long [2]

initiator cookie (8 bytes). host byte order

CookieR u_long [2]

responder cookie (8 bytes). host byte order

VALUES

IKE_SA kbuf A kbuf storing the fwisakmp_sa structure.

Flags uint currently only 2 flags are defined: (PEER_MOBILE, INITIATOR); used so Enforcement Module does not need to retrieve the whole kbuf from the kernel if we only want to know if this SA was established with a mobile user

RenegotiationTime uint renegotiation time of the SA

EXAMPLE-------- IKE_SA_table --------dynamic, id 77, attributes: keep, sync, expires 3600, limit 25000, hashsize 512, kbuf 1, free function

Cisco Anyconnect sample config

config twebvpnsvc image disk0:/anyconnect-win-2.0.0343-k9.pkg 1! this is a customerized vpn profile, if client does not needed, you can remove the following line using cisco default! svc profiles VitalProf disk0:/vpn-vig-tdc.xml

Page 15: VPN Troubleshooting for Checkpoint

tunnel-group-list enableenable outsidesvc enableexitip local pool SSLClientPool 192.168.100.1-192.168.100.50 mask 255.255.255.0access-list NONAT extended permit ip 192.168.5.0 255.255.255.0 192.168.100.0 255.255.255.0access-list vpnssl-split extended permit ip 192.168.5.0 255.255.255.0 192.168.100.0 255.255.255.0nat (inside) 0 access-list NONATusername userA password test123username userA attributesservice-type remote-accessexitusername userB password test12345username userB attributesservice-type remote-accessexitgroup-policy SSLCLientPolicy internalgroup-policy SSLCLientPolicy attributesdns-server value 192.168.1.51 192.168.1.61wins-server value 192.168.1.51 192.168.1.61address-pools value SSLClientPoolsplit-tunnel-policy tunnelspecifiedsplit-tunnel-network-list value vpnssl-splitwebvpnvpn-tunnel-protocol svcsvc keep-installer installed!svc profiles value VitalProfexit

sysopt connection permit-vpntunnel-group SSLClientProfile type remote-accesstunnel-group SSLClientProfile general-attributesdefault-group-policy SSLCLientPolicytunnel-group SSLClientProfile webvpn-attributesgroup-alias SSLVPNClient enableexitwr memwr stand

debug commandsh vpn-sessiondb svc,please be noticed, the default license for asa for web vpn or ssl vpn is only 2, you need to notify the client for this license limitation

Viewing all hidden commands on a Cisco ASA

The "show parser dump all" command will display all valid commands, including undocumented commands.The full output of the command came out to over 300 pages in MS WORD in 11 font.

A few examples of hidden VPN commands are as follows:clear/show ipsec sa counters clear/show crypto ipsec sa map clear/show crypto protocol statistics ssl clear/show crypto protocol statistics ssh clear/show crypto protocol statistics srtp clear/show crypto protocol statistics other clear/show crypto protocol statistics all clear/show vpn-sessiondb statistics

clear/show vpn-sessiondb statistics debug vpnlb debug vpn-sessiondb debug crypto isakmp timers debug crypto ca messages debug crypto condition peer subnet

debug crypto condition peer subnetdebug crypto condition peer

debug crypto condition user debug crypto condition unmatched isakmp debug crypto condition error ipsec debug crypto condition error isakmp

Page 16: VPN Troubleshooting for Checkpoint

http://www.netleets.com/search/label/tcpdump

Encryption Failure: Packet Is Dropped as There Is No Valid SAYou might see this error message when both ends of the VPN do not have the same definition for the encryption domain. First ensure that both ends of the VPN are defined with the same encryption domain. If they are the same, you should create objects that are exactly the same size as what is created on the remote end.

One annoying behavior FireWall-1 NG exhibits that FireWall-1 4.1 and earlier did not is the automatic simplification of subnets in IPSec SAs. For example, if your encryption domain contains explicit objects for 192.168.0.0/24 and 192.168.1.0/24, Check Point would attempt to negotiate an IPSec SA with 192.168.0.0/23 instead of generating SAs based on the network objects you created. To eliminate this behavior, use dbedit to make the following changes on your management console (see FAQ 4.2 for details on editing objects_5_0.C):

dbedit> modify properties firewall_properties ike_use_largest_possible_subnets falsedbedit> update properties firewall_properties

You must then reload the security policy for this change to take effect.

VPN Fails When Transferring Large PacketsSome applications set the Don't Fragment bit on certain packets. When the IPSec headers are added to the already large packet, the packet requires fragmentation in order to pass through the firewall. When Check Point creates the IPSec packet, the Don't Fragment bit from the original packet is maintained. FireWall-1 creates a fragmented packet that has the Don't Fragment bit set, so it cannot be fragmented and thus gets dropped at the next router.

You can force FireWall-1 to clear the Don't Fragment bit by changing the ipsec_dont_fragment property in objects_5_0.C to false. You can do this with the following commands in dbedit on the management console (craig is the firewall in this example):

dbedit> modify network_objects craig VPN:ipsec_dont_fragment falsedbedit> update network_objects craig

Alternatively, you can use the GUIdbedit tool to change the parameter. Either way, you must then reinstall the security policy for this change to take effect.

Debugging Interoperability Issues with IKEEveryone has a different interpretation about how to follow standards. As a result, when third-party products talk to one another, communication doesn't always work. One way to debug is to turn on IKE debugging.

In FireWall-1 4.1, it was necessary to stop and restart FireWall-1 in order to enable debugging. In NG, you can enable this on the firewall module with a simple command: vpn debug ikeon. You can also disable it with vpn debug ikeoff. When you enable debugging, $FWDIR/log/ike.elg gets created. This file contains the results of all IKE negotiations that occur. This file is a little difficult to read on its own. Fortunately, Check Point has a tool called IKEView that allows you to view this file in a more readable form. Unfortunately, it is available only to Check Point Certified Service Partners.

Check Point Logging Troubleshooting Guide

Page 17: VPN Troubleshooting for Checkpoint

Are the logs being sent to the manager?

Ok, so first of all are the logs being sent to the Smart Centre Manager or the necessary Log Manager ? We can check this by confirming whether the gateway is sending the log packets via the FW Log port tcp/257 upon the gateway and the manager. To do this use either or both of the following commands,

netstat -an | grep 257 - This will show the state of the TCP sockets.

tcpdump -ni [interface name] port 257 - This will show a packet capture of the FW Log packets on the subsequent interface.

If the gateway is not sending the logs then this can be down to one of the following issues,

1. SIC is not established.

2. The Logging configuration for the Gateway is not configured correctly.

3. The SmartCentre/Log Manager is not listening on port tcp/257.

4. There is an issue with FWD on the gateway. In some instances you may need to restart FWD via a cpstart. Though the root cause could be down to a number of factors.

Why are the logs not being displayed within SmartView tracker ?

Ok so the manager is receiving the logs but you may still not see them within the SmartView tracker this will be down to either the FWD (Firewall Daemon) or the log files being corrupted.

Log Files Corrupted

If the log files are corrupted you should expect to see no logs within the SmartView Tracker. If this is the case you will need to action the following steps :

1. Close the Log Viewer/SmartView Tracker and Policy Editor/SmartDashboard.

2. Execute the fwstop or cpstop command (depending on the version) from the command line.

3. Remove all files starting with fw.log and fw.logptr from the $FWDIR\log directory.

4. Execute the fwstart or cpstart (depending on the version) command.Full details can be found at Check Points KB within Solution ID sk6432.

Only some of the logs are not being displayed

If only some of the logs are not being displayed then this could point to an issue with the trust between the manager and the gateway.To confirm the issue you will need to debug FWD using the following steps.

root@cp-mgnt# fw debug fwd on TDERROR_ALL_ALL=5root@cp-mgnt# tail -f $FWDIR/log/fwd.elgroot@cp-mgnt# tail -f $FWDIR/log/fwd.elg | grep -i "Certificate is revoked" root@cp-mgnt# fw debug fwd off

Within these steps we first enable the debug. Then we run a live tail on the log file. And then we run a grep on the live tail for a specific error. The live tail allows us to view the end of the log file in real time. We finally turn off the debug.

Below shows an example of an error with the SIC trust between the Gateway and Manager obtained from the $FWDIR/log/fwd.elg,

[FWD 2177 1]@cp-mgnt[22 Jan 14:47:32] fwCert_ValCerts: Certificate is revoked. CN=cp-fw1,O=cp-mgnt..bizt7z[FWD 2177 1]@cp-mgnt[22 Jan 14:47:41] fwCert_ValCerts: Certificate is revoked. CN=cp-fw2,O=cp-mgnt..bizt7z

In this instance resetting SIC would resolve this issue

Page 18: VPN Troubleshooting for Checkpoint

Troubleshooting Checkpoint ClusterXL

I recently came across an issue where SmartView Monitor showed an error for ClusterXL on a freshly rebuilt Checkpoint IP565 firewall. Both Synchronization and Filter were stuck in an initilizing state, we tried the following troubleshooting steps initially to no avail:

5. cphastop followed by cphastart

6. cpstop followed by cpstart

7. reboot of the affected firewall

On digging deeper we noticed that one of the firewall devices was configured to use multicast and one for broadcast cluster communications, this was identified using the following command 'cphaprob -a if' which presents the following output:

eth-s1p3c0 non sync(non secured)eth-s4p3c0 non sync(non secured)eth-s4p4c0 non sync(non secured)eth-s1p1c0 non sync(non secured)eth-s1p4c0 sync(secured), multicasteth-s1p2c0 non sync(non secured)eth-s4p1c0 non sync(non secured)eth-s4p2c0 non sync(non secured)

Virtual cluster interfaces: 7

eth-s1p3c0 xx.xx.xx.xxeth-s4p3c0 xx.xx.xx.xxeth-s4p4c0 xx.xx.xx.xxeth-s1p1c0 xx.xx.xx.xxeth-s1p2c0 xx.xx.xx.xxeth-s4p1c0 xx.xx.xx.xxeth-s4p2c0 xx.xx.xx.xx

Both firewalls must be configured to use the same method of communication, which can be changed using the following command 'cphaconf set_ccp multicast' or 'cphaconf set_ccp broadcast'. Providing your switching infrastructure supports multicast you should use this mode due to the performance overhead of broadcast communication. This command failed to change the method of communication and left us with no other option than to perform the following steps:

1. Set Checkpoint Packages as in-active, then delete them ensuring that the Connectra package is removed first.

2. Re-install the Checkpoint R65 IPSO Wrapper

3. Re-install HFA 70

4. Re-establish SIC via CPConfig and SmartDashboard

5. Unassign and re-assign license via SmartUpdate

6. Push policy from the SmartDashboard

After performing thse steps the cluster CCP was back to multicast (bizare really...). We had to perform a reboot of the second device once this was completed, at which point both nodes of the cluster reported no ClusterXL errors, 'cphaprob list' showed the following output:

Page 19: VPN Troubleshooting for Checkpoint

# cphaprob list

Registered Devices:

Device Name: SynchronizationRegistration number: 0Timeout: noneCurrent state: OKTime since last report: 213003 sec

Device Name: FilterRegistration number: 1Timeout: noneCurrent state: OKTime since last report: 213003 sec

Device Name: cphadRegistration number: 2Timeout: 5 secCurrent state: OKTime since last report: 0.7 sec

Device Name: fwdRegistration number: 3Timeout: 5 secCurrent state: OKTime since last report: 0.5 sec

'fw ctl pstat' should also list the Synch as 'Able to Send/Receive sync packets' :

# fw ctl pstat

Machine Capacity Summary:Memory used: 14% (90MB out of 637MB) - below low watermarkConcurrent Connections: 26% (17876 out of 67900) - below low watermarkAggressive Aging is in monitor only

Hash kernel memory (hmem) statistics:Total memory allocated: 200278016 bytes in 48894 4KB blocks using 2 poolsInitial memory allocated: 20971520 bytes (Hash memory extended by 179306496 bytes)Memory allocation limit: 536870912 bytes using 10 poolsTotal memory bytes used: 23487660 unused: 176790356 (88.27%) peak: 34170776Total memory blocks used: 7126 unused: 41768 (85%) peak: 9164Allocations: 1183931215 alloc, 0 failed alloc, 1183678473 free

System kernel memory (smem) statistics:Total memory bytes used: 250335916 peak: 300842432Blocking memory bytes used: 1865892 peak: 2596156Non-Blocking memory bytes used: 248470024 peak: 298246276Allocations: 160033475 alloc, 0 failed alloc, 160032829 free, 0 failed free

Kernel memory (kmem) statistics:Total memory bytes used: 73389696 peak: 101169940Allocations: 1184023246 alloc, 0 failed alloc, 1183769860 free, 0 failed freeExternal Allocations: 0 for packets, 0 for SXL

Kernel stacks:0 bytes total, 0 bytes stack size, 0 stacks,0 peak used, 0 max stack bytes used, 0 min stack bytes used,0 failed stack calls

Page 20: VPN Troubleshooting for Checkpoint

INSPECT:1029526467 packets, -2128289516 operations, 373013811 lookups,2035 record, 183665476 extract

Cookies:-1649393933 total, 0 alloc, 0 free,4607 dup, -1525329462 get, 138972711 put,-1565092568 len, 217535 cached len, 0 chain alloc,0 chain free

Connections:54513276 total, 52537755 TCP, 1898998 UDP, 76506 ICMP,17 other, 49485065 anticipated, 1 recovered, 17882 concurrent,24286 peak concurrent

Fragments:213594 fragments, 105472 packets, 389 expired, 0 short,0 large, 0 duplicates, 0 failures

NAT:23444077/0 forw, 29804768/0 bckw, 53234829 tcpudp,14016 icmp, 702040-723136 alloc

Sync:Version: newStatus: Able to Send/Receive sync packetsSync packets sent:total : 78286072, retransmitted : 16171, retrans reqs : 20, acks : 3Sync packets received:total : 17030603, were queued : 16591, dropped by net : 15retrans reqs : 8840, received 3 acksretrans reqs for illegal seq : 0dropped updates as a result of sync overload: 0

Check Point Firewall-1Useful Firewall-1 command line utilities:

Unload current security policy

fw unloadlocal

VPN Tunnel command line access (e.g. delete SAs)

vpn tu

Display overlapping VPN Encryption Domains

vpn overlap_encdom [communities|traditional]

List current Firewall interfaces

fw ctl iflist

Show HA / ClusterXL state

cpstat ha

cphaprob state

Page 21: VPN Troubleshooting for Checkpoint

cphastop / cphastart

Display State of Checkpoint HA Interfaces

cphaprob -a if

Stop/Start Checkpoint HA/ClusterXL

cphastop / cphastart

Display State of Checkpoint HA Interfaces

cphaprob -a if

Manually failover

cphaprob -d STOP -s problem -t 0 register

cphaprob list

cphaprob -d STOP unregister

Display State of ClusterXL IGMP

cphaprob stat (Notify if IGMP membership is supported)

cphaprob igmp (Display the current IGMP membership settings)

SmartCenterBackup and Restore SmartCenter

upgrade_export

$FWDIR/bin/upgrade_tools/upgrade_import

Check whether licensed for management high availability (Management HA)

cplic check mgmtha

SecurePlatformSecurePlatform configuration commands:

Configure Interfaces, Routes etc

sysconfig

Add static routes

config route add dest 192.168.1.0/24 via 192.168.0.1 dev eth0 metric 0 s-persistant on apply on

Configure Network Interfaces

config conn help

Page 22: VPN Troubleshooting for Checkpoint

config conn set name eth1 type eth onboot on iff-up on local 192.168.1.2/24 broadcast 192.168.1.255 s-persistant on s-code up mtu 1500

Configure Bonded Network Interfaces (NIC Team, 2 physical, 1 logical interface)

config conn add name bond0 type bond onboot on iff-up on mtu 1500 bond-mode active-backup bond-miimon 100 bond-downdelay 200 bond-updelay 200 bond-primary eth1 local 192.168.1.2/24

config conn add name eth1 type eth onboot on iff-up on mtu 1500 master-bond bond0

config conn add name eth4 type eth onboot on iff-up on mtu 1500 master-bond bond0

Useful SecurePlatform command line utilities:

Enter OS commands

expert

Assign interfaces to correct physical NICs

(Edit /etc/sysconfig/ethtab)

[Expert@FIREWALL]# cat ethtab

eth0 00:21:5A:27:DC:E6

eth1 00:21:5A:27:DC:E4

eth2 00:1F:29:5C:82:F5

Set Kernel parameters

(Edit $FWDIR/boot/modules/fwkern.conf)

fwha_mac_magic=0×11

fwha_mac_forward_magic=0×10

fwha_monitor_if_link_state=1

fwha_enable_igmp_snooping=1

fwha_igmp_version=2

Flag disconnected NICs

echo eth6 >> $FWDIR/conf/discntd.if

Show status of Bonded Network Interfaces

cphaconf show_bond -a

Display Versions

SPLAT: ver

Firewall: fw ver

Page 23: VPN Troubleshooting for Checkpoint

Performance Pack: sim ver –k

Linux: uname -a

Change shell to permit WinSCP connection

usermod -s /bin/bash fwadmin

Change shell timout (cpshell)

idle mm where mm = timeout in minutes (permanent change, updates /etc/cpshell/cpshell.state and is passed on to expert shell)

Change shell timout (bash)

TMOUT = ss where ss = timeout in minutes

export TMOUT

Display the number of CPUs presented to SecurePlatform OS

grep ‘physical id’ /proc/cpuinfo|wc -l

Display the CoreXL CPU Affinity

fw ctl affinity -l

Advanced Routing (gated) Commands

ps -eaf | grep gated

cpwd_admin list


Recommended