The pfSense Store

Author Topic: Packets disappear between vpn client and vpn server  (Read 100 times)

0 Members and 1 Guest are viewing this topic.

Offline FuriousGeorge

  • Newbie
  • *
  • Posts: 23
  • Karma: +0/-0
    • View Profile
Packets disappear between vpn client and vpn server
« on: December 29, 2017, 04:23:35 pm »
I have a hub and spoke site-to-site vpn setup, however -- only in one direction and only on one spoke -- nodes on the client lan  cannot reach nodes on the server lan, although they can reach the server itself.  The opposite direction works fine (i.e. nodes on server lan can reach nodes on this client-lan.

The really odd thing is that the problem is not between the vpn server and the node on its lan.  I know this because when I attempt to reach the server from the client-node, I can see the packets reach the server in the tcp dump, however if I try to get beyond the server to one of the nodes on the server-lan, the packets appear to vanish on the vpn client, never making it to the server.

Here is a diagram I made of the situation:

Code: [Select]
   A ====> B ====> C ====> D  ssh from node on server lan to node on client lan works, as does any portion of that
   A       B <==== C <==== D  ssh from node on client lan to vpn server works
   A <==== B <==== C       D  ssh from vpn client to node on server lan works
   A XXXXX B XXXXX C <==== D  ssh from node on client lan to node on server lan fails!!!  packets disappear between ovpnc and ovpns interfaces
       
Server    VPN     VPN    Client
 LAN     Server  Client   LAN

While it would seem this has to be a firewall / NAT problem, I've been over my rules time and again, as well as logs (no blocks or rejections), and I've pfctl -d on both vpn server and client.

Note that another VPN client lan and vpn client node exists, E and F, which work fine in both directions.

Here is the routing table on the vpn server:

Code: [Select]
: netstat -r
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            65.145.227.209     UGS      vtnet0
10.0.0.0/24        link#7             U       bridge0
10.0.0.1           link#7             UHS         lo0
10.5.0.0/24        10.0.0.100         UGS     bridge0
10.10.0.0/24       10.0.0.101         UGS     bridge0
10.20.0.0/24       10.0.0.102         UGS     bridge0
63.141.227.208/29  link#1             U        vtnet0
sangat.fortulite.n link#1             UHS         lo0
localhost          link#4             UH          lo0

... and on the vpn client ...

Code: [Select]
: netstat -r
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            lo0-100.NWRKNJ-VFT UGS         em1
10.0.0.0/24        link#8             U        ovpnc1
10.0.0.102         link#8             UHS         lo0
10.5.0.0/24        10.0.0.100         UGS      ovpnc1
10.10.0.0/24       10.0.0.101         UGS      ovpnc1
10.20.0.0/24       link#1             U           em0
pfSense            link#1             UHS         lo0
10.250.0.0/24      10.255.255.1       UGS      ovpnc2
10.250.0.0/16      10.255.255.1       UGS      ovpnc2
10.255.255.0/24    10.255.255.1       UGS      ovpnc2
10.255.255.1       link#9             UH       ovpnc2
10.255.255.4       link#9             UHS         lo0
nsphil01.verizon.n d6:1b:af:2b:e3:98  UHS         em1
nsnwrk01.verizon.n d6:1b:af:2b:e3:98  UHS         em1
93.232.81.0/24     link#2             U           em1
pool-96-242-81-51. link#2             UHS         lo0
localhost          link#3             UH          lo0



for reference, the is a vpn-client that does not have the problem:

Code: [Select]
netstat -r
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            lo0-100.NWRKNJ-VFT UGS         em1
10.0.0.0/24        link#8             U        ovpnc1
10.0.0.100         link#8             UHS         lo0
10.5.0.0/24        link#1             U           em0
ads-120elmst-pfsen link#1             UHS         lo0
10.10.0.0/24       10.0.0.101         UGS      ovpnc1
10.20.0.0/24       10.0.0.102         UGS      ovpnc1
10.30.0.0/24       10.0.0.103         UGS      ovpnc1
10.250.0.0/24      10.255.255.1       UGS      ovpnc2
10.250.0.0/16      10.255.255.1       UGS      ovpnc2
10.255.255.0/24    10.255.255.1       UGS      ovpnc2
10.255.255.1       link#9             UH       ovpnc2
10.255.255.2       link#9             UHS         lo0
nsphil01.verizon.n 48:5d:36:85:1e:48  UHS         em1
nsnwrk01.verizon.n 48:5d:36:85:1e:48  UHS         em1
93.242.143.0/24    link#2             U           em1
pool-96-242-148-23 link#2             UHS         lo0
localhost          link#3             UH          lo0

The node on the client lan is set up to use the vpn-client as router/gateway:

Code: [Select]
# ip route list
default via 10.20.0.1 dev vmbr0
10.20.0.0/24 dev vmbr0  proto kernel  scope link  src 10.20.0.251

However, the node on the server lan has it's own wan, and also the routes must be statically set up (no way to push from client to server, afaik):

Code: [Select]
# ip route list
default via 63.141.227.209 dev vmbr0 onlink
10.0.0.0/24 dev vmbr1 proto kernel scope link src 10.0.0.250
10.5.0.0/24 via 10.0.0.1 dev vmbr1
10.10.0.0/24 via 10.0.0.1 dev vmbr1
10.20.0.0/24 via 10.0.0.1 dev vmbr1
66.141.226.208/29 dev vmbr0 proto kernel scope link src 63.141.227.210

I'm starting to lean toward a bug somewhere, rather than a mistake in my implementation.  I'm at a loss, and any help is appreciated.
« Last Edit: December 29, 2017, 04:40:45 pm by FuriousGeorge »