pfSense Gold Subscription

Author Topic: pfSense 2.2 not passing traffic, but ping does get through  (Read 15251 times)

0 Members and 1 Guest are viewing this topic.

Offline jpsense42

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
pfSense 2.2 not passing traffic, but ping does get through
« on: January 30, 2015, 09:46:10 am »
Hi,

I have a fresh 2.2 install on a KVM guest. The host has two physical network interfaces (bridges), one LAN, one connected to my ISP router (LAN side of the router, so pfSense WAN side is a private network). pfSense has two virtio interfaces. I understand this setup causes double NAT, but I have no choice, my ISP does not allow other equipment or config changes to their router.

The following fails:
- Clients cannot reach the internet, no traffic gets passed. Ping DOES work however, see below!
- pfSense console: telnet <ISP router LAN ip> 80 > no connection, seems pfSense itself cannot do anything but ping hosts
- pfSense console: telnet <any webserver> 80 > no connection

The following all works:
- I can reach the webconfig via the LAN
- LAN clients can ping everything, using pfSense as their gateway, all the way to 8.8.8.8 (!)
- pfSense can ping everything, all the way to 8.8.8.8 and everything on the LAN
- Clients connected directly to the ISP router can access the internet just fine

The following config checks look OK to me:
- Firewall rules are clean, default (allow LAN to access everything. automatic NAT)
- Private networks are not blocked, bogon networks are not blocked on the WAN interface
- Gateway (ISP router LAN side) is the default and only gateway and is UP
- Double-checked IP addresses, subnet masks, gateways address.

Version: 2.2-RELEASE, built on Thu Jan 22 14:04:25 CST 2015

Offline Trel

  • Sr. Member
  • ****
  • Posts: 368
  • Karma: +11/-1
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #1 on: January 30, 2015, 10:47:38 am »
Are you able to ping by hostname or only by IP?
(from both client machines and pfsense)

Offline jpsense42

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #2 on: January 30, 2015, 11:42:44 am »
Both. pfSense is able to resolve host names using 8.8.8.8. DNS lookup via the web configurator works.
If I configure a LAN client to use pfSense as a local DNS server, lookups work too. So clients can ping google.com and they get replies.

This is a copy/paste from diagnostics / DNS lookup:

Resolve DNS hostname or IP
Hostname or IP   
    =    216.58.219.174
 
Resolution time per server   
Server   Query time
127.0.0.1   5 msec
8.8.8.8   107 msec
8.8.4.4   87 msec
More Information:   Ping
Traceroute
NOTE: The following links are to external services, so their reliability cannot be guaranteed.

IP WHOIS @ DNS Stuff
IP Info @ DNS Stuff

Offline jpsense42

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #3 on: January 30, 2015, 11:45:28 am »
And this is a copy/paste from Diagnostics / Ping:

PING google.com (216.58.219.78): 56 data bytes
64 bytes from 216.58.219.78: icmp_seq=0 ttl=55 time=55.964 ms
64 bytes from 216.58.219.78: icmp_seq=1 ttl=55 time=55.682 ms
64 bytes from 216.58.219.78: icmp_seq=2 ttl=55 time=56.017 ms

--- google.com ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 55.682/55.888/56.017/0.147 ms

Offline fragged

  • Sr. Member
  • ****
  • Posts: 402
  • Karma: +27/-1
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #4 on: January 30, 2015, 12:01:12 pm »
So you can ping things by both IP and hostname -> everything is working? What is the actual issue here?

Offline jpsense42

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #5 on: January 30, 2015, 12:04:16 pm »
If I set pfSense as my default gateway, I can ping, but I cannot access anything else. No ssh, no http. I was hoping pfSense would allow me to do more than just ping ;-)

Edit: I just realized, ssh and http are TCP. Is it possible pfSense is passing UDP (DNS) and ICMP (ping) but not TCP?
« Last Edit: January 30, 2015, 12:12:40 pm by jpsense42 »

Offline fragged

  • Sr. Member
  • ****
  • Posts: 402
  • Karma: +27/-1
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #6 on: January 30, 2015, 12:28:59 pm »
Assuming you have the default "Default allow LAN to any rule" on LAN then it should pass everything through from LAN -> WAN. Check the firewall log and do a packet capture to see whats going on.

Offline doktornotor

  • Hero Member
  • *****
  • Posts: 8553
  • Karma: +956/-278
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #7 on: January 30, 2015, 12:35:04 pm »
Instead of letting people guess, why don't you just check/post firewall logs/rules here?
Do NOT PM for help!

Offline jpsense42

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #8 on: January 31, 2015, 04:01:23 pm »
These (see attachment) are the LAN firewall rules. The WAN tab is empty.
Text version:


ID   Proto      Source  Port    Destination     Port    Gateway  Queue   Schedule         Description
     *          *       *       LAN Address     80      *        *                       Anti-Lockout Rule
     IPv4*      LAN net *       *               *       *        none                    Default allow LAN to any rule
     IPv6*      LAN net *       *               *       *        none                    Default allow LAN IPv6 to any rule




Offline jpsense42

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #9 on: January 31, 2015, 04:21:46 pm »
As for the firewall log, it's empty. It did contain IP6 related entries. I disabled IP6 on the client, cleared the log and tried connecting to the internet via pfSense again. Nothing appeared in the log.

The system dashboard shows:
2.2-RELEASE (i386)
built on Thu Jan 22 14:04:25 CST 2015
FreeBSD 10.1-RELEASE-p4
Unable to check for updates.

pfSense is unable to check for updates, although it can ping the world, resolve host names and it's connected to a router that allows full internet access - clients connected directly to the ISP router can connect to anything on the internet.

I think: no TCP routing is happening here. Or it is happening, but no NAT is happening. The ISP router would not know what to do with non-NATted packets from the pfSense LAN side.

Offline jpsense42

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #10 on: January 31, 2015, 04:23:22 pm »
This is wat the NAT outbound rules look like. It's a fresh install, no changes.

Offline jpsense42

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #11 on: January 31, 2015, 04:38:31 pm »
Packet captures below. I can't see what's wrong from this, at least not yet.

Packet capture, LAN side.
Client 192.168.0.196 is trying to access Google (216.58.219.78:80)
pfSense LAN side is 192.168.0.252

Code: [Select]
19:06:53.285513 IP 192.168.0.252.53 > 192.168.0.196.60003: UDP, length 76
19:06:54.714668 IP 192.168.0.196.51332 > 216.58.219.78.80: tcp 0
19:06:54.841045 IP 192.168.0.196.56142 > 192.168.0.252.53: UDP, length 37
19:06:54.845216 IP 192.168.0.252.53 > 192.168.0.196.56142: UDP, length 77
19:06:54.846406 IP 192.168.0.196.51338 > 216.58.219.78.443: tcp 0
19:06:54.951584 IP 192.168.0.196.51333 > 216.58.219.78.80: tcp 0
19:06:55.291665 IP 192.168.0.196.51336 > 216.58.219.78.443: tcp 0
19:06:55.455828 IP 192.168.0.196.51337 > 216.58.219.78.443: tcp 0
19:06:56.671783 IP 192.168.0.196.51334 > 62.69.175.109.80: tcp 0
19:06:56.809020 IP 192.168.0.196.52478 > 192.168.0.252.53: UDP, length 28
19:06:56.810237 IP 192.168.0.252.53 > 192.168.0.196.52478: UDP, length 44
19:06:56.811612 IP 192.168.0.196.51339 > 216.58.219.78.80: tcp 0
19:06:57.062611 IP 192.168.0.196.51340 > 216.58.219.78.80: tcp 0


Same scenario, now WAN side. pfSense WAN is 192.168.7.252, ISP router is 192.168.7.254

Code: [Select]
19:07:51.898694 IP 192.168.7.252.27180 > 205.251.192.57.53: UDP, length 44
19:07:52.198428 IP 205.251.192.57.53 > 192.168.7.252.27180: UDP, length 196
19:07:52.204635 IP 192.168.7.252.56383 > 108.160.165.189.443: tcp 0
19:07:52.210622 IP 192.168.7.252 > 192.168.7.254: ICMP echo request, id 14886, seq 35773, length 44
19:07:52.211711 IP 192.168.7.254 > 192.168.7.252: ICMP echo reply, id 14886, seq 35773, length 44
19:07:53.220143 IP 192.168.7.252 > 192.168.7.254: ICMP echo request, id 14886, seq 36029, length 44
19:07:53.220916 IP 192.168.7.254 > 192.168.7.252: ICMP echo reply, id 14886, seq 36029, length 44
19:07:53.748700 IP 192.168.7.252.9196 > 216.58.219.78.443: tcp 0
19:07:54.008374 IP 192.168.7.252.38630 > 216.58.219.78.443: tcp 0
19:07:54.228310 IP 192.168.7.252 > 192.168.7.254: ICMP echo request, id 14886, seq 36285, length 44
19:07:54.229275 IP 192.168.7.254 > 192.168.7.252: ICMP echo reply, id 14886, seq 36285, length 44
19:07:55.201675 IP 192.168.7.252.56383 > 108.160.165.189.443: tcp 0
19:07:55.267333 IP 192.168.7.252 > 192.168.7.254: ICMP echo request, id 14886, seq 36541, length 44
19:07:55.268015 IP 192.168.7.254 > 192.168.7.252: ICMP echo reply, id 14886, seq 36541, length 44
19:07:56.307582 IP 192.168.7.252 > 192.168.7.254: ICMP echo request, id 14886, seq 36797, length 44
19:07:56.308639 IP 192.168.7.254 > 192.168.7.252: ICMP echo reply, id 14886, seq 36797, length 44
19:07:56.712085 IP 192.168.7.252.24188 > 108.160.169.188.80: tcp 0
19:07:57.039836 IP 192.168.7.252.25088 > 62.69.166.210.80: tcp 0
19:07:57.303517 IP 192.168.7.252.27234 > 216.239.32.10.53: UDP, length 39
19:07:57.307442 IP 192.168.7.252 > 192.168.7.254: ICMP echo request, id 14886, seq 37053, length 44
19:07:57.308079 IP 192.168.7.254 > 192.168.7.252: ICMP echo reply, id 14886, seq 37053, length 44
19:07:57.388742 IP 216.239.32.10.53 > 192.168.7.252.27234: UDP, length 44
19:07:57.394470 IP 192.168.7.252.8493 > 216.58.219.78.80: tcp 0
19:07:57.646223 IP 192.168.7.252.49477 > 216.58.219.78.80: tcp 0
19:07:58.320115 IP 192.168.7.252 > 192.168.7.254: ICMP echo request, id 14886, seq 37309, length 44
19:07:58.320916 IP 192.168.7.254 > 192.168.7.252: ICMP echo reply, id 14886, seq 37309, length 44
19:07:59.330378 IP 192.168.7.252 > 192.168.7.254: ICMP echo request, id 14886, seq 37565, length 44
19:07:59.331114 IP 192.168.7.254 > 192.168.7.252: ICMP echo reply, id 14886, seq 37565, length 44
19:07:59.743873 IP 192.168.7.252.9196 > 216.58.219.78.443: tcp 0
19:08:00.003967 IP 192.168.7.252.38630 > 216.58.219.78.443: tcp 0
19:08:00.339999 IP 192.168.7.252 > 192.168.7.254: ICMP echo request, id 14886, seq 37821, length 44
19:08:00.340785 IP 192.168.7.254 > 192.168.7.252: ICMP echo reply, id 14886, seq 37821, length 44
19:08:00.393862 IP 192.168.7.252.8493 > 216.58.219.78.80: tcp 0
19:08:00.640923 IP 192.168.7.252.49477 > 216.58.219.78.80: tcp 0
19:08:00.647629 IP 192.168.7.252.33846 > 216.239.34.10.53: UDP, length 48


Offline cmb

  • Hero Member
  • *****
  • Posts: 11230
  • Karma: +893/-7
    • View Profile
    • Chris Buechler
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #12 on: January 31, 2015, 04:49:12 pm »
Guessing some kind of checksum offloading problem. Which type of NICs are you using in KVM? Try disabling hardware checksum offloading under System>Advanced, Misc. Reboot afterwards to be on the safe side.

Offline jpsense42

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #13 on: January 31, 2015, 05:31:37 pm »
I found the setting under System > Advanced > Networking.

Disable hardware checksum offload
Checking this option will disable hardware checksum offloading. Checksum offloading is broken in some hardware, particularly some Realtek cards. Rarely, drivers may have problems with checksum offloading and some specific NICs.


I selected the checkbox and rebooted. No change. The pfSense VM can ping external hosts, but ssh from the pfSense console to an external ssh server does not work, clients cannot access the internet via pfSense.
These are my KVM NIC settings (virtio). br0 is LAN on the host, br1 is WAN.

Code: [Select]
<interface type='bridge'>
      <mac address='54:52:00:44:13:69'/>
      <source bridge='br0'/>
      <target dev='vnet4'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='54:52:00:1d:48:7e'/>
      <source bridge='br1'/>
      <target dev='vnet5'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface>

Offline Bullz3y3

  • Newbie
  • *
  • Posts: 3
  • Karma: +2/-0
    • View Profile
Re: pfSense 2.2 not passing traffic, but ping does get through
« Reply #14 on: February 22, 2015, 11:21:02 pm »
I had this same problem with ProxmoVE. pfSense installed as KVM with "VirtIO" emulator which is default for KVM. WAN bride with eth0 and go out. Local bridge for LAN side of pfSense.

Installed Windows & Ubuntu with VirtIO Driver. When Windows VM was set to go through pfSense I could ping but no internet no TCP/UDP connections at all. Same scenario. After bashing my head on the wall for whole sleepless night trying to resolve this. Finally I decided to setup XenServer instead of Proxmox which runs Xen hypervisor.

Implemented the same setup in XenServer with all default settings. Windows was installed with default Realtek NIC driver. Alverything worked perfectly fine.

When I installed xe-tools which turned Realtek NIC to "Xen Paravirtualized driver" it stopped work with same results as above. When I uninstalled xe-tools it worked again.

Conclusion
From this what I can see is Paravirtualzied drives are causing this issue in both setup. VirtIO in KVM & PV in Xen. With other NIC emulators like e1000 or Realtek it works fine.

I haven't found a solution to get this working with para drivers which will improve the performance.