pfSense Support Subscription

Author Topic: LAN clients on KVM bridge can't connect through firewall  (Read 159 times)

0 Members and 1 Guest are viewing this topic.

Offline gpw928

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
    • View Profile
LAN clients on KVM bridge can't connect through firewall
« on: November 19, 2017, 01:19:04 am »

Hi,

I have pfSense installed as a guest under KVM running on Debian 9.2.

There is one other active guest, running FreeBSD 11.1.

The KVM server has a a software bridge at 192.168.1.26/24.  It's associated with a NIC on that subnet:

Code: [Select]
auto eno1
iface eno1 inet manual

auto br0
iface br0 inet static
        address 192.168.1.26
        netmask 255.255.255.0
        gateway 192.168.1.254
        bridge_ports eno1
        bridge_stp off
        bridge_maxwait 0
        bridge_fd 0

The pfSense LAN is on a virtual NIC connected to the bridge on 192.168.1.0/24.

This is what things look like on the KVM server:

Code: [Select]
[orac.1145] $ ifconfig eno1 
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 50:46:5d:76:25:9b  txqueuelen 1000  (Ethernet)
        RX packets 3131064  bytes 666408021 (635.5 MiB)
        RX errors 0  dropped 40512  overruns 0  frame 0
        TX packets 2085772  bytes 2071053850 (1.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xf7c00000-f7c20000 

[orac.1146] $ ifconfig br0   
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.26  netmask 255.255.255.0  broadcast 192.168.1.255
        ether 50:46:5d:76:25:9b  txqueuelen 1000  (Ethernet)
        RX packets 2932839  bytes 567227483 (540.9 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 874493  bytes 1931119811 (1.7 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[orac.1147] $ ifconfig vnet3
vnet3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether fe:54:00:de:4a:fe  txqueuelen 1000  (Ethernet)
        RX packets 26190  bytes 17428082 (16.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 35805  bytes 4633540 (4.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

On the pfSense KVM guest, the network looks like this (vtnet0 is unused -- it was the "dummy WAN" initially, before ppp was configured).

Code: [Select]
[2.4.0-RELEASE][admin@pfSense.my.domain]/root: ifconfig -a
vtnet0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 52:54:00:85:d0:2f
        hwaddr 52:54:00:85:d0:2f
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
vtnet1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6c00bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 52:54:00:de:4a:fe
        hwaddr 52:54:00:de:4a:fe
        inet6 fe80::5054:ff:fede:4afe%vtnet1 prefixlen 64 scopeid 0x2
        inet 192.168.1.37 netmask 0xffffff00 broadcast 192.168.1.255
        inet 192.168.1.254 netmask 0xffffffff broadcast 192.168.1.254
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
enc0: flags=0<> metric 0 mtu 1536
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: enc
pflog0: flags=100<PROMISC> metric 0 mtu 33160
        groups: pflog
pfsync0: flags=0<> metric 0 mtu 1500
        groups: pfsync
        syncpeer: 224.0.0.240 maxupd: 128 defer: on
        syncok: 1
ppp0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
        inet 121.44.11.88 --> 10.64.64.0  netmask 0xffffffff
        inet6 fe80::c88c:8100:109:bd37%ppp0 prefixlen 64 scopeid 0x7
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Noth that the default route for the 192.168.1.0/24 network is at 192.168.1.254 and an IP alias on vtnet1.

The WAN is connected to a ppp link on a 3G USB modem.  No known problems.

The other active KVM guest is connected to the same bridge as the LAN at 192.168.1.36.

The packet filter rules look like this ("pfctl -s all"):
Code: [Select]
TRANSLATION RULES:
no nat proto carp all
nat-anchor "natearly/*" all
nat-anchor "natrules/*" all
nat on ppp0 inet from 127.0.0.0/8 to any port = isakmp -> 121.44.11.88 static-port
nat on ppp0 inet from 192.168.1.0/24 to any port = isakmp -> 121.44.11.88 static-port
nat on ppp0 inet from 127.0.0.0/8 to any -> 121.44.11.88 port 1024:65535
nat on ppp0 inet from 192.168.1.0/24 to any -> 121.44.11.88 port 1024:65535
no rdr proto carp all
rdr-anchor "relayd/*" all
rdr-anchor "tftp-proxy/*" all
rdr on ppp0 inet proto tcp from any to 121.44.11.88 port = 32022 -> 192.168.1.24 port 22
rdr-anchor "miniupnpd" all

FILTER RULES:
scrub on ppp0 all fragment reassemble
scrub on vtnet1 all fragment reassemble
anchor "relayd/*" all
anchor "openvpn/*" all
anchor "ipsec/*" all
block drop in log quick inet from 169.254.0.0/16 to any label "Block IPv4 link-local"
block drop in log quick inet from any to 169.254.0.0/16 label "Block IPv4 link-local"
block drop in log inet all label "Default deny rule IPv4"
block drop out log inet all label "Default deny rule IPv4"
block drop in log inet6 all label "Default deny rule IPv6"
block drop out log inet6 all label "Default deny rule IPv6"
pass quick inet6 proto ipv6-icmp all icmp6-type unreach keep state
pass quick inet6 proto ipv6-icmp all icmp6-type toobig keep state
pass quick inet6 proto ipv6-icmp all icmp6-type neighbrsol keep state
pass quick inet6 proto ipv6-icmp all icmp6-type neighbradv keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type echorep keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type routersol keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type routeradv keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type neighbrsol keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type neighbradv keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type echorep keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type routersol keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type routeradv keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type neighbrsol keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type neighbradv keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type echoreq keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type routersol keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type routeradv keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type neighbrsol keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type neighbradv keep state
pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type echoreq keep state
pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type routersol keep state
pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type routeradv keep state
pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type neighbrsol keep state
pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type neighbradv keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type echoreq keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type routersol keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type routeradv keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type neighbrsol keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type neighbradv keep state
block drop log quick inet proto tcp from any port = 0 to any label "Block traffic from port 0"
block drop log quick inet proto udp from any port = 0 to any label "Block traffic from port 0"
block drop log quick inet proto tcp from any to any port = 0 label "Block traffic to port 0"
block drop log quick inet proto udp from any to any port = 0 label "Block traffic to port 0"
block drop log quick inet6 proto tcp from any port = 0 to any label "Block traffic from port 0"
block drop log quick inet6 proto udp from any port = 0 to any label "Block traffic from port 0"
block drop log quick inet6 proto tcp from any to any port = 0 label "Block traffic to port 0"
block drop log quick inet6 proto udp from any to any port = 0 label "Block traffic to port 0"
block drop log quick from <snort2c> to any label "Block snort2c hosts"
block drop log quick from any to <snort2c> label "Block snort2c hosts"
block drop in log quick proto carp from (self) to any
pass quick proto carp all no state
block drop in log quick proto tcp from <sshlockout> to (self) port = ssh label "sshlockout"
block drop in log quick proto tcp from <webConfiguratorlockout> to (self) port = https label "webConfiguratorlockout"
block drop in log quick from <virusprot> to any label "virusprot overload table"
block drop in log quick on ppp0 from <bogons> to any label "block bogon IPv4 networks from WAN"
block drop in log quick on ppp0 from <bogonsv6> to any label "block bogon IPv6 networks from WAN"
block drop in log on ! ppp0 inet from 121.44.11.88 to any
block drop in log inet from 121.44.11.88 to any
block drop in log on ppp0 inet6 from fe80::c88c:8100:109:bd37 to any
block drop in log quick on ppp0 inet from 10.0.0.0/8 to any label "Block private networks from WAN block 10/8"
block drop in log quick on ppp0 inet from 127.0.0.0/8 to any label "Block private networks from WAN block 127/8"
block drop in log quick on ppp0 inet from 172.16.0.0/12 to any label "Block private networks from WAN block 172.16/12"
block drop in log quick on ppp0 inet from 192.168.0.0/16 to any label "Block private networks from WAN block 192.168/16"
block drop in log quick on ppp0 inet6 from fc00::/7 to any label "Block ULA networks from WAN block fc00::/7"
block drop in log on ! vtnet1 inet from 192.168.1.0/24 to any
block drop in log inet from 192.168.1.37 to any
block drop in log inet from 192.168.1.254 to any
block drop in log on vtnet1 inet6 from fe80::5054:ff:fede:4afe to any
pass in quick on vtnet1 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server"
pass in quick on vtnet1 inet proto udp from any port = bootpc to 192.168.1.37 port = bootps keep state label "allow access to DHCP server"
pass out quick on vtnet1 inet proto udp from 192.168.1.37 port = bootps to any port = bootpc keep state label "allow access to DHCP server"
pass in on lo0 inet all flags S/SA keep state label "pass IPv4 loopback"
pass out on lo0 inet all flags S/SA keep state label "pass IPv4 loopback"
pass in on lo0 inet6 all flags S/SA keep state label "pass IPv6 loopback"
pass out on lo0 inet6 all flags S/SA keep state label "pass IPv6 loopback"
pass out inet all flags S/SA keep state allow-opts label "let out anything IPv4 from firewall host itself"
pass out inet6 all flags S/SA keep state allow-opts label "let out anything IPv6 from firewall host itself"
pass out route-to (ppp0 10.64.64.0) inet from 121.44.11.88 to ! 121.44.11.88 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
pass in quick on vtnet1 proto tcp from any to (vtnet1) port = https flags S/SA keep state label "anti-lockout rule"
pass in quick on vtnet1 proto tcp from any to (vtnet1) port = http flags S/SA keep state label "anti-lockout rule"
pass in quick on vtnet1 proto tcp from any to (vtnet1) port = ssh flags S/SA keep state label "anti-lockout rule"
anchor "userrules/*" all
pass in quick on vtnet1 inet from 192.168.1.0/24 to any flags S/SA keep state label "USER_RULE: Default allow LAN to any rule"
pass in quick on ppp0 reply-to (ppp0 10.64.64.0) inet proto tcp from any to 192.168.1.24 port = ssh flags S/SA keep state label "USER_RULE: NAT External ssh access to ding"
anchor "tftp-proxy/*" all
No queue in use

STATES:
vtnet1 udp 255.255.255.255:7423 <- 192.168.1.2:1035       NO_TRAFFIC:SINGLE
vtnet1 udp 255.255.255.255:7423 <- 192.168.1.3:46996       NO_TRAFFIC:SINGLE
ppp0 icmp 121.44.11.88:16960 -> 192.231.203.3:16960       0:0
vtnet1 udp 224.0.0.251:5353 <- 192.168.1.35:5353       NO_TRAFFIC:SINGLE
vtnet1 tcp 192.168.1.37:22 <- 192.168.1.145:22038       ESTABLISHED:ESTABLISHED
vtnet1 tcp 150.101.195.202:80 <- 192.168.1.10:49690       CLOSING:ESTABLISHED
ppp0 tcp 121.44.11.88:16478 (192.168.1.10:49690) -> 150.101.195.202:80       ESTABLISHED:CLOSING
ppp0 udp 121.44.11.88:123 -> 203.135.184.123:123       MULTIPLE:SINGLE
ppp0 udp 121.44.11.88:123 -> 13.55.50.68:123       MULTIPLE:SINGLE
ppp0 udp 121.44.11.88:123 -> 27.124.125.250:123       MULTIPLE:SINGLE
vtnet1 tcp 111.221.29.104:443 <- 192.168.1.7:49882       ESTABLISHED:ESTABLISHED
ppp0 tcp 121.44.11.88:1113 (192.168.1.7:49882) -> 111.221.29.104:443       ESTABLISHED:ESTABLISHED
vtnet1 tcp 104.98.26.36:443 <- 192.168.1.10:49689       CLOSING:ESTABLISHED
ppp0 tcp 121.44.11.88:37824 (192.168.1.10:49689) -> 104.98.26.36:443       ESTABLISHED:CLOSING
vtnet1 tcp 23.40.74.230:443 <- 192.168.1.10:49692       CLOSING:ESTABLISHED
ppp0 tcp 121.44.11.88:33607 (192.168.1.10:49692) -> 23.40.74.230:443       ESTABLISHED:CLOSING
vtnet1 tcp 17.252.252.41:443 <- 192.168.1.10:49693       ESTABLISHED:ESTABLISHED
ppp0 tcp 121.44.11.88:8945 (192.168.1.10:49693) -> 17.252.252.41:443       ESTABLISHED:ESTABLISHED
vtnet1 tcp 23.40.74.230:443 <- 192.168.1.10:49694       CLOSING:ESTABLISHED
ppp0 tcp 121.44.11.88:13807 (192.168.1.10:49694) -> 23.40.74.230:443       ESTABLISHED:CLOSING
vtnet1 udp 202.127.210.37:123 <- 192.168.1.147:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:25400 (192.168.1.147:123) -> 202.127.210.37:123       SINGLE:NO_TRAFFIC
vtnet1 udp 103.242.68.69:123 <- 192.168.1.36:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:21197 (192.168.1.36:123) -> 103.242.68.69:123       SINGLE:NO_TRAFFIC
vtnet1 udp 192.231.203.132:123 <- 192.168.1.26:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:57787 (192.168.1.26:123) -> 192.231.203.132:123       SINGLE:NO_TRAFFIC
vtnet1 udp 103.214.220.220:123 <- 192.168.1.36:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:19718 (192.168.1.36:123) -> 103.214.220.220:123       SINGLE:NO_TRAFFIC
vtnet1 udp 202.127.210.37:123 <- 192.168.1.27:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:23807 (192.168.1.27:123) -> 202.127.210.37:123       SINGLE:NO_TRAFFIC
vtnet1 udp 103.239.8.22:123 <- 192.168.1.36:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:41032 (192.168.1.36:123) -> 103.239.8.22:123       SINGLE:NO_TRAFFIC
vtnet1 udp 255.255.255.255:67 <- 0.0.0.0:68       NO_TRAFFIC:SINGLE
vtnet1 tcp 172.217.17.46:443 <- 192.168.1.145:17495       ESTABLISHED:ESTABLISHED
ppp0 tcp 121.44.11.88:54736 (192.168.1.145:17495) -> 172.217.17.46:443       ESTABLISHED:ESTABLISHED
vtnet1 tcp 172.217.17.46:443 <- 192.168.1.145:17496       ESTABLISHED:ESTABLISHED
ppp0 tcp 121.44.11.88:8140 (192.168.1.145:17496) -> 172.217.17.46:443       ESTABLISHED:ESTABLISHED
vtnet1 udp 150.203.22.28:123 <- 192.168.1.36:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:43931 (192.168.1.36:123) -> 150.203.22.28:123       SINGLE:NO_TRAFFIC
vtnet1 udp 122.252.184.186:123 <- 192.168.1.36:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:51527 (192.168.1.36:123) -> 122.252.184.186:123       SINGLE:NO_TRAFFIC
vtnet1 udp 150.203.1.10:123 <- 192.168.1.36:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:23369 (192.168.1.36:123) -> 150.203.1.10:123       SINGLE:NO_TRAFFIC
vtnet1 udp 203.0.178.191:123 <- 192.168.1.36:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:20164 (192.168.1.36:123) -> 203.0.178.191:123       SINGLE:NO_TRAFFIC
ppp0 udp 121.44.11.88:123 -> 203.122.222.149:123       MULTIPLE:SINGLE
ppp0 udp 121.44.11.88:123 -> 121.0.0.42:123       MULTIPLE:SINGLE
ppp0 udp 121.44.11.88:123 -> 203.217.19.190:123       MULTIPLE:SINGLE
ppp0 udp 121.44.11.88:123 -> 103.214.220.220:123       MULTIPLE:SINGLE
vtnet1 udp 192.168.1.255:137 <- 192.168.1.7:137       NO_TRAFFIC:SINGLE
vtnet1 udp 224.0.0.252:5355 <- 192.168.1.7:61576       NO_TRAFFIC:SINGLE
vtnet1 udp 202.127.210.37:123 <- 192.168.1.36:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:48946 (192.168.1.36:123) -> 202.127.210.37:123       SINGLE:NO_TRAFFIC
vtnet1 udp 224.0.0.252:5355 <- 192.168.1.7:58273       NO_TRAFFIC:SINGLE
vtnet1 udp 203.12.160.2:123 <- 192.168.1.36:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:55055 (192.168.1.36:123) -> 203.12.160.2:123       SINGLE:NO_TRAFFIC
vtnet1 udp 203.0.178.191:123 <- 192.168.1.147:123       SINGLE:MULTIPLE
ppp0 udp 121.44.11.88:4996 (192.168.1.147:123) -> 203.0.178.191:123       MULTIPLE:SINGLE
vtnet1 udp 150.203.22.28:123 <- 192.168.1.27:123       SINGLE:MULTIPLE
ppp0 udp 121.44.11.88:30245 (192.168.1.27:123) -> 150.203.22.28:123       MULTIPLE:SINGLE
vtnet1 udp 203.0.178.191:123 <- 192.168.1.27:123       SINGLE:MULTIPLE
ppp0 udp 121.44.11.88:51593 (192.168.1.27:123) -> 203.0.178.191:123       MULTIPLE:SINGLE
vtnet1 udp 203.12.160.2:123 <- 192.168.1.147:123       SINGLE:MULTIPLE
ppp0 udp 121.44.11.88:8374 (192.168.1.147:123) -> 203.12.160.2:123       MULTIPLE:SINGLE
vtnet1 udp 239.255.255.250:1900 <- 192.168.1.3:39102       NO_TRAFFIC:SINGLE
vtnet1 udp 203.26.24.6:123 <- 192.168.1.36:123       NO_TRAFFIC:SINGLE
ppp0 udp 121.44.11.88:27473 (192.168.1.36:123) -> 203.26.24.6:123       SINGLE:NO_TRAFFIC

INFO:
Status: Enabled for 0 days 01:18:06           Debug: Urgent

Interface Stats for vtnet1            IPv4             IPv6
  Bytes In                         2810962            59578
  Bytes Out                        8089433                0
  Packets In
    Passed                           22366               94
    Blocked                             12              533
  Packets Out
    Passed                           16747                0
    Blocked                              0                0

State Table                          Total             Rate
  current entries                       67               
  searches                           98715           21.1/s
  inserts                            10645            2.3/s
  removals                           10578            2.3/s
Counters
  match                              11431            2.4/s
  bad-offset                             0            0.0/s
  fragment                               0            0.0/s
  short                                  0            0.0/s
  normalize                              0            0.0/s
  memory                                 0            0.0/s
  bad-timestamp                          0            0.0/s
  congestion                             0            0.0/s
  ip-option                              0            0.0/s
  proto-cksum                            0            0.0/s
  state-mismatch                         0            0.0/s
  state-insert                           0            0.0/s
  state-limit                            0            0.0/s
  src-limit                              0            0.0/s
  synproxy                               0            0.0/s
  map-failed                             0            0.0/s

LABEL COUNTERS:
Block IPv4 link-local 11420 0 0 0 0 0 0 0
Block IPv4 link-local 3861 0 0 0 0 0 0 0
Default deny rule IPv4 3861 251 13429 251 13429 0 0 0
Default deny rule IPv4 10825 0 0 0 0 0 0 0
Default deny rule IPv6 11421 533 54050 533 54050 0 0 0
Default deny rule IPv6 7560 0 0 0 0 0 0 0
Block traffic from port 0 11358 0 0 0 0 0 0 0
Block traffic from port 0 9646 0 0 0 0 0 0 0
Block traffic to port 0 10827 0 0 0 0 0 0 0
Block traffic to port 0 9646 0 0 0 0 0 0 0
Block traffic from port 0 11359 0 0 0 0 0 0 0
Block traffic from port 0 533 0 0 0 0 0 0 0
Block traffic to port 0 533 0 0 0 0 0 0 0
Block traffic to port 0 533 0 0 0 0 0 0 0
Block snort2c hosts 11358 0 0 0 0 0 0 0
Block snort2c hosts 11358 0 0 0 0 0 0 0
sshlockout 11358 0 0 0 0 0 0 0
webConfiguratorlockout 696 0 0 0 0 0 0 0
virusprot overload table 4398 0 0 0 0 0 0 0
block bogon IPv4 networks from WAN 4398 0 0 0 0 0 0 0
block bogon IPv6 networks from WAN 243 0 0 0 0 0 0 0
Block private networks from WAN block 10/8 241 0 0 0 0 0 0 0
Block private networks from WAN block 127/8 241 0 0 0 0 0 0 0
Block private networks from WAN block 172.16/12 241 0 0 0 0 0 0 0
Block private networks from WAN block 192.168/16 241 0 0 0 0 0 0 0
Block ULA networks from WAN block fc00::/7 241 0 0 0 0 0 0 0
allow access to DHCP server 4132 45 14760 45 14760 0 0 22
allow access to DHCP server 0 0 0 0 0 0 0 0
allow access to DHCP server 10043 0 0 0 0 0 0 0
pass IPv4 loopback 10804 46 5025 23 1553 23 3472 23
pass IPv4 loopback 46 0 0 0 0 0 0 0
pass IPv6 loopback 579 0 0 0 0 0 0 0
pass IPv6 loopback 23 0 0 0 0 0 0 0
let out anything IPv4 from firewall host itself 11336 46 5025 23 3472 23 1553 23
let out anything IPv6 from firewall host itself 6964 0 0 0 0 0 0 0
let out anything from firewall host itself 6964 38388 10795851 19201 8898594 19187 1897257 6943
anti-lockout rule 11345 1759 633803 832 152875 927 480928 6
anti-lockout rule 10 0 0 0 0 0 0 0
anti-lockout rule 10 837 110758 320 35324 517 75434 2
USER_RULE: Default allow LAN to any rule 11336 33368 9515823 18065 1982752 15303 7533071 3557
USER_RULE: NAT External ssh access to ding 7244 0 0 0 0 0 0 0

TIMEOUTS:
tcp.first                   120s
tcp.opening                  30s
tcp.established           86400s
tcp.closing                 900s
tcp.finwait                  45s
tcp.closed                   90s
tcp.tsdiff                   30s
udp.first                    60s
udp.single                   30s
udp.multiple                 60s
icmp.first                   20s
icmp.error                   10s
other.first                  60s
other.single                 30s
other.multiple               60s
frag                         30s
interval                     10s
adaptive.start            58200 states
adaptive.end             116400 states
src.track                     0s

LIMITS:
states        hard limit    97000
src-nodes     hard limit    97000
frags         hard limit     5000
table-entries hard limit   200000

TABLES:
bogons
bogonsv6
snort2c
sshlockout
virusprot
webConfiguratorlockout

OS FINGERPRINTS:
758 fingerprints loaded

The WAN and LAN firewall rules are "standard" (there is one extra rule for port-forwarding inbound ssh connections, with is untested).

The KVM server (192.168.1.26) and other guest (192.168.1.36) can't traverse the firewall. 

Everything else seems to work fine.

All clues gratefully accepted...

Offline gpw928

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
    • View Profile
Re: LAN clients on KVM bridge can't connect through firewall
« Reply #1 on: November 25, 2017, 09:47:10 pm »
Hi,

I have looked at the packet traces from multiple perspectives.

The KVM bridge connects the KVM host and all its guests (including the pfSense firewall) to the LAN (192.168.1.0/24).

The default gateway on the LAN is 192.168.1.254, and that's an IP alias on the pfSense LAN interface (192.168.1.37).

The pfSense firewall works perfectly for all hosts not associated with the KVM bridge.

Hosts associated with the KVM bridge (i.e. the KVM server itself, and all its guests except the pfSense firewall itself) can't establish connections through the pfSense firewall.  Tracing a TCP connection from a KVM guest to an Internet host shows the initial SYN entering the pfSense firewall via the default route, and being re-transmitted out the WAN interface.  The SYN is then re-transmitted several times before the connection times out.  The expected SYN/ACK response never happens.

This feels very much like NAT (or masquerading) is not turned on for those hosts on the KVM bridge.  The NAT table looks like this:

Code: [Select]
[2.4.2-RELEASE][admin@pfSense.oakes.consulting]/root: pfctl -s nat
no nat proto carp all
nat-anchor "natearly/*" all
nat-anchor "natrules/*" all
nat on ppp0 inet from 127.0.0.0/8 to any port = isakmp -> 121.44.4.84 static-port
nat on ppp0 inet from 192.168.1.0/24 to any port = isakmp -> 121.44.4.84 static-port
nat on ppp0 inet from 127.0.0.0/8 to any -> 121.44.4.84 port 1024:65535
nat on ppp0 inet from 192.168.1.0/24 to any -> 121.44.4.84 port 1024:65535
no rdr proto carp all
rdr-anchor "relayd/*" all
rdr-anchor "tftp-proxy/*" all
rdr on ppp0 inet proto tcp from any to 121.44.4.84 port = 32022 -> 192.168.1.24 port 22
rdr-anchor "miniupnpd" all

My setup mst be very similar to that used by ProxMox users. 

Any ideas appreciated.

Cheers,

Offline gpw928

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
    • View Profile
Re: LAN clients on KVM bridge can't connect through firewall
« Reply #2 on: December 02, 2017, 06:03:37 pm »
Hi,

Does anyone have pfSense working under KVM?

I need to figure out if I have a configuration issue which can be resolved or something more difficult.

Cheers,