Post on 11-Dec-2021
transcript
Network TroubleshootingSave Time and Solve Problems Faster with AccessEnforcer
Jun. 27, 2019
Today’s Speakers
Duane PettusSupport DirectorCalyptix Security
Warren ParkerTraining DirectorCalyptix Security
Adam SuttonMarketing Director
Calyptix Security
• Tools for troubleshooting• Troubleshooting
Slow internet RDP not accessible Internet not accessible CalyptixVPN not connecting CalyptixVPN disconnects unexpectedly DVR, Camera, etc. not accessible Email not passing Website not accessible
• Q + A
Agenda
Tools for Troubleshooting
• Info needed- Username and password for AccessEnforcer- Network config. (internal and external)
• Tools in AccessEnforcer- Link Status- Outbound Filtering Page- ARP Table- Packet Analyzer- Ping- DNS Lookup- Telnet
Tools for Troubleshooting
• Third-Party Tools- Wireshark- IPChicken.com- Putty- www.subnet-calculator.com- MXtoolbox.com- Downdetector.com- Remote Monitoring Service
• Additional Tools When On-Site- Laptop - Serial cable (DB9 TO RJ45)- USB adapter (optional)
Tools for Troubleshooting
Network Troubleshooting with AccessEnforcer
Where is the break in communication?
Troubleshooting
Issue: Slow internet
Slow internet
• Scenario- Client complains the internet is slow.
Slow internet
• Support Team Thoughts- ISP issues
- Is the installation new? - Have any ISP settings changed?
- Firmware update- Did the AccessEnforcer update recently?
- Bandwidth hog- Is a device on the network hogging
bandwidth? - For example, did someone turn on the
backup service early? - AccessEnforcer settings
- Has the configuration changed recently?
Slow internet
• Diagnose- Ping the IP address of the modem
- use -t for continuous- Check the network outages in the area
- https://downdetector.com/- Check for these issues in AccessEnforcer
- Link Status - Duplexing issues
- IDS/IPS- Outbound scanning
- ARP table- Device in front of the AccessEnforcer is reachable
- Route table (PPPOE)- IP address of device in front of the AE is listed
- Outbound filtering- Status going in and out
Slow internet
• Diagnose (cont.)- Live connections
- State count - If the number is high,
- High Graph Utilization- Run a packet capture to determine which
device is maxing out bandwidth- Network alerts
- Port 53 blocked (means Snort is restarting)- Run packet captures
Issue: RDP not working
• Scenario- New AccessEnforcer installed- Port-forwarding to RDP fails
• Support Team Thoughts: - Is another firewall blocking the
connection?- Windows firewall- Modem
RDP not working
• Diagnose - Probably an issue with Windows Firewall- Check Port Forwarding to ensure it’s
working properly- Go to Home > Diagnostics > Ping
- Ping the machine locally- Go to Home > Diagnostics > Packet Capture
- Capture packets on Host IP and port 3389- Go to Home > Diagnostics > Telnet
- Telnet to the internal IP address and port 3389
- Ensure the machine’s gateway points to the AccessEnforcer
RDP not working
Issue: Internet not accessible
Internet not accessible
• Scenario- Big storm last night. Comcast came,
replaced the modem, and tested that it could reach the internet.
- Now they have left and I cannot reach the internet.
Internet not accessible
• Support Team Thoughts:- You said BIG STORM, right?- Are power issues causing the
problem? - Did a power surge blow a network
port? - Switch, AccessEnforcer, Modem
- Did the AccessEnforcer lose power when writing to a critical or database file?
Internet not accessible
• Diagnose- Boot the AE with the serial cable and
see if you can get to a login prompt- Could be the WAN port or one of the
other ports is blown or missing- Check Link Status for all ports
- If no IP address, the device will not pass traffic without an IP
- Check if you were in Bridge Mode- Check if Comcast put the modem back to
defaults and you are now handed an internal IP address
Internet not accessible
• Diagnose (cont.)- Check if the Modem’s firewall is
enabled- Test the ports by taking the cable out
and watching the link status - Link light will go out and when you plug it
back in should go green- Check the ARP table for WAN IP
address and Gateway WAN IP Address.
Issue: CalyptixVPN not connecting
• Scenario- CalyptixVPN client fails to make a
connection
CalyptixVPN not connecting
• Support Team Thoughts- Is this the first CalyptixVPN client for this
site?- If not, are others able to connect?- If not, did they work previously?
- AccessEnforcer configuration correct? - CalyptixVPN enabled? - External IP address correct under Setup > VPN
> CalpytixVPN Settings?- CalyptixVPN configuration correct?
- Correct IP address in configuration? - TAP Adapter working properly?
CalyptixVPN not connecting
• Diagnose- Check the IP address on the client
- Right click the red circle and go to “edit Config”.
- See if it matches the public IP of the AccessEnforcer
- Check the Setup > VPN > CalyptixVPN page and make sure it is enabled and has the correct external IP address.
- Check the Home > CalyptixVPN Login attempts and see if clients have recently logged into the AccessEnforcer
- Check the Packet capture and see if you see traffic from the source IP address.
- If not, it is probably the TAP adapter
- If so, then click the Apply button on the CalyptixVPN settings page
CalyptixVPN not connecting
Issue: CalyptixVPN disconnects
CalyptixVPN disconnects
• Scenario- Calyptix VPN connects, but the
connection stops after a short while
• Support Team Thoughts - Is the same client trying to connect
from two locations at once (duplicate connection)?
CalyptixVPN disconnects
• Diagnose:- Check Home > CalyptixVPN Login
attempts- The Client number is generally logging in
an out- This generally means there is a duplicate client
connected at the same time. - I have seen where a client at the same location
has connected with a computer upstairs and is trying to login with their laptop downstairs with the same client.
- Create a new client called (Name) Laptop or (Name) phone and use that specific for that device
Issue: DVR, Camera, Etc. Not Connecting
• Scenario- Switching from another UTM firewall
and now the DVR does not work.
• Support Team Thoughts:- DVR
- Is the gateway correct?
- Port Forwarding- Is it configured correctly?
DVR, Camera, Etc. Not Connecting
• Diagnose- Gather information about the DVR
and what changed the other firewall- Log into AccessEnforcer
- Port Forwarding- Check the configuration
- Perform two packet captures- Try to reach the device from the LAN- Try to reach the device from another network- Check the capture for one-way or two-way
traffic- Look at the ports in the capture for clues
DVR, Camera, Etc. Not Connecting
Issue: Email not passing
Email not passing
• Scenario- Emails are no longer being sent or
received.
Email not passing
• Support Team Thoughts: - Is email entering the network? - Is email exiting the network? - Is email hosted behind the AccessEnforcer? - Does a third-party service filter spam before
forwarding to the network? - If so, is it on the Dynamic Blacklist?
- Are you getting a bounce back messages?- Did the ISP block port 25 on the modem?
Email not passing
• Diagnose: Inbound- Check the SMTP Rate limiter- Check a bounce back message against
the portal article https://online.calyptix.com/smtperror
- Check MXtoolbox and make sure that the MX record did not change
- Check the Port forwarding page that it did not stop taking traffic
- Setup the Packet captures and watch the traffic all the way through
Email not passing
• Diagnose: Outbound- Go to the Advanced page and make
sure that Only allow these SMTP … is not limited
- Hit the apply button on the Advanced page
- Watch the traffic to see if it gets to us on both the LAN and WAN
Issue: Website not accessible
• Scenario- Client cannot open a website but is
able to open others.- The client can open the website on
another network (one not behind an AccessEnforcer).
Website not accessible
• Support Team Thoughts:- Is something blocking access to the
website? - AccessEnforcer – IDS/IPS, Web Filter- ISP
- Is something blocking your IP? - Site host - Content hosting service (Akamai, AWS,
Microsoft)
- What firmware version is on the AccessEnforcer?
Website not accessible
• Diagnose:- Log into the AccessEnforcer
- Dynamic Blacklist
- Try to access the site from behind a different AccessEnforcer
- If the site opens, then the cause is unlikely to be a bug in the firewall
- Perform two packet captures- Get the website’s IP address (via DNS
lookup)- Watch the traffic
Website not accessible
Online Portal
• First place you should check for information on AccessEnforcer
• Location: https://online.calyptix.com/
• Resources include: - AccessEnforcer Handbook
- Training videos
- Release notes
- Price list, deals, and more
• Questions and issues we resolve often become Portal articles.
• Trouble accessing the portal? Contact support@calyptix.com
Online Portal
Partner Deals
Safer for the Storms Ahead!• Big discounts on AccessEnforcer with 1 year
of services, 3 years of warranty
• Free month of service on all monthly firewalls deployed this month when you deploy 3 or more
See the deals:• Go to https://online.calyptix.com• Click “Partner Deals”
QUESTIONS?
Bonus: IPsec VPN Problems
• Scenario- IPsec VPN traffic is slow
• Mitigation: - Whitelist the internal IP range and the Peer IP address of
the remote site. The IDS/IPS will not inspect these packets so the speed through tunnel should increase.
• Steps- Log into AccessEnforcer- Go to Security > Network > Static Whitelist- Add the remote peer’s local range and the remote peer’s
IP address
IPsec VPN Problems
• Scenario- IPsec VPN tunnel went down unexpectedly and will not come
back up.
• Possible cause: - Remote peer’s external IP address was added to the Dynamic
Blacklist.
• Steps- Log into AccessEnforcer
- Go to Security > Network > Static Blacklist
- Check if the remote peer’s IP is listed
- If so, remove the IP from the list
- Go to Security > Network > Static Whitelist
- Add the remote peer’s external IP address
IPsec VPN Problems
• Scenario- AccessEnforcer will not connect to my Cisco
• Possible cause: - Misconfiguration of one or more of the devices.
• Steps: - In AccessEnforcer, the Phase 2's Diffie Hellman
Group must be set to None- Follow the instructions shown on this Online
Portal article: https://online.calyptix.com/node/224
IPsec VPN Problems
• Scenario- AccessEnforcer will not connect to my RV042/RV082
• Possible cause- The Linksys RV042 / RV082 does not support automatic keying with
the AccessEnforcer.
- There also appears to be a bug in the RV042 / RV082 GUI where the Incoming SPI will only accept a 7-character SPI value at most.
• Note- AccessEnforcer allows 7-character SPI’s to inter-operate with
Linksys devices.
• Mitigation- Set a manual keying policy instead.
IPsec VPN Problems
Bonus: Lost Email
• Scenario- Client did not receive a specific email.
• Steps- Confirm the sender is using correct email
address- Check bounce errors
- Check if sender received a bounce message
- Troubleshoot error messages - Identify the exact phrase/number of the error message- Identify the source of the error message (use Google)- Look to the error source for fastest resolution
Lost Email
• Common email bounce errors from AccessEnforcer
- 554 Your IP address ... is not allowed. (Error LR-55443.)
- Cause: IP address is on the SMTP Rate Limiter
- 554 Your IP address ... is not allowed. (Error OEG-55768.)
- Cause: IP address is on the Geographical Policy
- 554 Your IP address ... is not allowed. (Error 86117.)
- Possible cause: IP address is on the Static or Dynamic Blacklist
- Possible cause: IP address does not resolve correctly with the FQDN check
Lost Email
• Common email bounce errors from AccessEnforcer (cont.)
- 552 Message size exceeds fixed maximum message size- Possible cause: Message size limits on Exchange server need to be
increased.
- 550 Message rejected as spam by Content Filtering.- Possible cause: Content filtering on Exchange server is set too high.
Remove the filter on the Exchange server. This is common on Exchange 2007 and higher.
- 503 Need mail command - Possible cause: This is a common message that one of your drives on the
Exchange server is out of space and cannot accept any more connectors.
• Complete list of SMTP errors: https://online.calyptix.com/node/253
Lost Email
• Check if the email is in the user’s quarantine- Go to Security > Email > Quarantine by Address- Click the user’s email address to open the
quarantine- Click “See All” on the top-right- See if the email is listed
- Message highlighting
- Red – error from SMTP server
- Dull Red – message flagged as a virus
- Blue – forwarded to the SMTP server
- White – held in spam filter
- Dark grey – deleted
Lost Email
• Check if the email is in the user’s quarantine (cont.)
- If the message is found and is safe- Click the “release” icon on the left to forward
the email to the user
- If the message is found and no safe- Click the email to preview- Click “Show reason” for more information- Click “SMTP Error” to reveal the exact error
message
Lost Email
Bonus: Squid Error
• Steps- Find the IP of the website- Check the Dynamic Blacklist
- Go to Security > Network > Dynamic Blacklist
- Check if the IP address is listed- If the address is listed:
- Remove the address from the Blacklist
- Go to Security > Network > Static Whitelist- Add the IP address to the Whitelist
Squid Error
• Steps (cont.)- Change settings for IDS/IPS rule
- Go to Home > Network Alerts- Look for the IP address- Click the checkbox next to the IP address- Click the drop-menu and select “Ignore the
Rule that cause alert”- Restart IDS/IPS to apply the changes
- Go to Security > Network > IDS/IPS Settings- Click “Apply”
Squid Error
Squid Error: Example
Squid Error: Example
Bonus: Serial Cable
Serial cable
• AccessEnforcer ships with DB9 to RJ45 cable
Serial cable
• Recommend adding R-232 to USB cable
Serial cable
• Why It’s Used- A serial connection allows you to access
the booting scripts that Calyptix engineers can use to determine problems with the device.
- This will help determine if the device is booting correctly and if the unit can access the login screen.
- This is essential in determining problems with a device to see if it should be replaced or returned through the RMA process.
Serial Cable
• Connecting- Download Putty.exe and install
(http://www.putty.org/ )- Open the application- Under “Connection Type” click
“Serial” and then “Open”- Use the serial cable to connect the
AccessEnforcer to your computer
Serial Cable