Netgate Store

Author Topic: Conenction to linux box dies over ipsec  (Read 217 times)

0 Members and 1 Guest are viewing this topic.

Offline jca1981

  • Jr. Member
  • **
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Conenction to linux box dies over ipsec
« on: January 24, 2018, 03:23:17 am »
set up a new department with new pfsense and at first it seemed all is working.
first i give will you a overview of our net structure, we have a server net 10.0.0.0/16 with a pfsense firewall and from there all departments are connected via internet over ipsec.
the new department is 10.5.0.0/16
all seamed to work fine at first until i set up my a unifi ap, i could not talk to the unifi controller on the server net, it gives timeout errors, i can ping unifi from the ap and ping the ap from the controller.
ive set any any rules on ipsec interface, seems nothing is blocked.
next thing i tried to ssh from a server 10.0.0.204 into a linux box 10.5.1.30 and ecery time i connect via ssh and run a command, eg ls the connection dies.
i have included wireshark dumps from both the 1.5.0.0 pfsense box and 10.0.0.0 pfsense box, where i try to ssh and run ls and it dies. i can see tcp retransmission errors in the logs.

hope someone will help, there is some free beer to the one who solves it :)
 
« Last Edit: January 25, 2018, 05:36:46 am by jca1981 »

Offline jca1981

  • Jr. Member
  • **
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: Conenction to linux box dies over ipsec
« Reply #1 on: January 25, 2018, 05:38:40 am »
could it be a MTU issue, or maybe MTU over ipsec?

Offline JKnott

  • Hero Member
  • *****
  • Posts: 1472
  • Karma: +65/-17
    • View Profile
Re: Conenction to linux box dies over ipsec
« Reply #2 on: January 25, 2018, 05:49:59 am »
could it be a MTU issue, or maybe MTU over ipsec?

IP is designed to tolerate different MTUs, especially with TCP, as would be used with SSH.  When a TCP connection is set up, the MTU is depended on the smallest MSS at either end.  Then, if a packet is still too small, it will be fragmented on IPv4.
This page unintentionally left blank.

Offline jca1981

  • Jr. Member
  • **
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: Conenction to linux box dies over ipsec
« Reply #3 on: January 25, 2018, 07:02:58 am »
soo, you are saying that it most likley is not MTU.
any idea what i could try to resolve it?

Offline JKnott

  • Hero Member
  • *****
  • Posts: 1472
  • Karma: +65/-17
    • View Profile
Re: Conenction to linux box dies over ipsec
« Reply #4 on: January 25, 2018, 11:07:38 am »
One very useful technique is to examine the traffic, to see what's happening.  This can be done the the pfSense packet capture or Wireshark.  Once you understand what the problem is and where, you can fix it.
This page unintentionally left blank.

Offline jca1981

  • Jr. Member
  • **
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: Conenction to linux box dies over ipsec
« Reply #5 on: January 25, 2018, 11:13:33 am »
yea, i have included downloaderble wireshark pcap in this thread, would you mind looking at them to see if you can figure out what is happening, ive tried and like i wrote i found retransmision and malformed packet, but i am not a wireshark master :)

Offline JKnott

  • Hero Member
  • *****
  • Posts: 1472
  • Karma: +65/-17
    • View Profile
Re: Conenction to linux box dies over ipsec
« Reply #6 on: January 25, 2018, 02:43:40 pm »
In the from 10.5.0.0 I see some 1512 byte packets, with a 1448 packet size.  What is the MTU on IPSec?  You can try pinging with different size packets, to see where it fails.  Use the -s option for this.  Also, I see the do not fragment flag is set, which means path MTU discovery is used, instead of fragmenting.  This should generate some "too big" ICMP messages, but I don't see those.  Have they been filtered out?

This page unintentionally left blank.

Offline jca1981

  • Jr. Member
  • **
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: Conenction to linux box dies over ipsec
« Reply #7 on: January 25, 2018, 03:06:05 pm »
im have not set mtu on ipsec so i guess it is using 1400, to set mtu on ipsec mss clamping needs to be enabled, right?
biggest MTU i can get now with ping -s is 1410.
ive now tried enabling "do not fragment" and "firewall scrub" under system/advanced/firewall & nat, not sure if there is a specific place only for ipsec.

Offline jca1981

  • Jr. Member
  • **
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: Conenction to linux box dies over ipsec (Solved)
« Reply #8 on: January 25, 2018, 03:58:08 pm »
ok, so tried setting mss clamping to 1300 and now it works, thanks, triet upping it to 1400 and it stopped working, found 1380 to be working after trying som more,

Offline JKnott

  • Hero Member
  • *****
  • Posts: 1472
  • Karma: +65/-17
    • View Profile
Re: Conenction to linux box dies over ipsec
« Reply #9 on: January 25, 2018, 04:00:30 pm »
Linux normally uses PMTUD to set packet size.  Do you see the ICMP "too big" messages?  I'm not sure about IPSec settings, as I haven't used IPSec with pfSense.  The MSS is normally used when setting up a TCP connection to tell the other end the maximum supported packet size.  It has nothing to do with any router, including pfSense.  It is PMTUD that's used to determine the maximum packet size that will fit the smallest MTU along the path.
This page unintentionally left blank.