The pfSense Store

Author Topic: bug in pf on freebsd 7  (Read 4894 times)

0 Members and 1 Guest are viewing this topic.

Offline crtek

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
bug in pf on freebsd 7
« on: December 19, 2008, 07:10:16 am »
I had a problem with creation of repl-to rules on wan in 1.2,
you can see my post here http://forum.pfsense.org/index.php/topic,13024.0.html.

Then I tried to upgrade to 1.2.1rc4
and I can see that those rules on wan are created correctly in this release

# User-defined rules follow
pass in quick on $wan reply-to (le0 x.x.x.x) from any to any keep state  label "USER_RULE"
pass in quick on $OPT1 reply-to (le2 y.y.y.y) from any to any keep state  label "USER_RULE"

but my pfsense did not reply back to source addresses like it should so i did some testing

I created a testing environment with 5 pcs

pc 1 ip 20.50.10.106/16 gw 20.0.0.101

pc2 ip1 10.50.0.1/16 gw 10.0.0.2
      ip2 20.50.10.105/16

pfsense wan ip 10.50.0.2/16 gw 10.0.0.1
           lan ip 192.168.233.2
           opt1 20.50.0.1/16 gw 20.0.0.3

pc3  ip 20.50.0.3/16 gw 20.0.0.1

pc4 ip 192.168.233.1/24 gw 192.168.233.2
 on pc4 i run firebird server just to have some service running

I loaded my custom pf.conf
that looks like this (only 4 rules)

rdr on le0 proto tcp from any to 10.50.0.2 port { 3050 } -> 192.168.233.1
rdr on le2 proto tcp from any to 20.50.0.2 port { 3050 } -> 192.168.233.1

pass  in  quick  on le0   reply-to ( le0 10.0.0.1 )from any to any keep state
pass  in  quick  on le2 reply-to ( le2 20.0.0.1 )  from any to any keep state


My test went like this
1. pfsense machine running pfsense 1.2
-loaded rules with this command pfctl -F all -f /root/pf.conf
-from pc1 tried to acces my service on pc4 and there was no problems
pc1 issued command:
#telnet 10.50.0.2 3050
Trying 10.50.0.2...
Connected to 10.50.0.2

pfsense issued command:# tcpdump -ni le0 | grep 3050
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on le0, link-type EN10MB (Ethernet), capture size 96 bytes
10:54:09.671074 IP 20.50.0.1.2657 > 10.50.0.2.3050: P 1712515217:1712515222(5) ack 804699406 win 16384 <nop,nop,timestamp 2445543632 0>
10:54:09.671276 IP 10.50.0.2.3050 > 20.50.0.1.2657: F 1:1(0) ack 5 win 65530 <nop,nop,timestamp 1176208 2445543632>
10:54:09.671490 IP 20.50.0.1.2657 > 10.50.0.2.3050: . ack 2 win 16384 <nop,nop,timestamp 2445543632 1176208>
10:54:09.671941 IP 20.50.0.1.2657 > 10.50.0.2.3050: F 5:5(0) ack 2 win 16384 <nop,nop,timestamp 2445543632 1176208>
10:54:09.672150 IP 10.50.0.2.3050 > 20.50.0.1.2657: . ack 6 win 65530 <nop,nop,timestamp 1176208 2445543632>
10:54:11.382724 IP 20.50.0.1.7338 > 10.50.0.2.3050: S 2478544458:2478544458(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 3331861149 0>
10:54:11.382864 IP 10.50.0.2.3050 > 20.50.0.1.7338: S 2788614409:2788614409(0) ack 2478544459 win 65535 <mss 1460,nop,wscale 0,nop,nop,timestamp 0 0,nop,nop,sackOK>
10:54:11.383167 IP 20.50.0.1.7338 > 10.50.0.2.3050: . ack 1 win 16384 <nop,nop,timestamp 3331861149 0>
10:54:14.057866 IP 20.50.0.1.7338 > 10.50.0.2.3050: P 1:6(5) ack 1 win 16384 <nop,nop,timestamp 3331861154 0>
10:54:14.058088 IP 10.50.0.2.3050 > 20.50.0.1.7338: F 1:1(0) ack 6 win 65530 <nop,nop,timestamp 1176251 3331861154>
10:54:14.058325 IP 20.50.0.1.7338 > 10.50.0.2.3050: . ack 2 win 16384 <nop,nop,timestamp 3331861154 1176251>
10:54:14.058559 IP 20.50.0.1.7338 > 10.50.0.2.3050: F 6:6(0) ack 2 win 16384 <nop,nop,timestamp 3331861154 1176251>
10:54:14.058631 IP 10.50.0.2.3050 > 20.50.0.1.7338: . ack 7 win 65530 <nop,nop,timestamp 1176251 3331861154>

# netstat -nrf inet
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            10.50.0.1          UGS         0       45    le0
10.50/16           link#1             UC          0        0    le0
10.50.0.1          00:0c:29:32:bc:0c  UHLW        2       70    le0    399
20.50/16           link#3             UC          0        0    le2
20.50.0.3          00:0c:29:6c:39:c3  UHLW        1       70    le2   1143
127.0.0.1          127.0.0.1          UH          0       28    lo0
192.168.233        link#2             UC          0        0    le1
192.168.233.1      00:50:56:c0:00:01  UHLW        1      104    le1    901


8801 packets received by filter
1839 packets dropped by kernel
All well than

2. pfsense machine running pfsense 1.2.1rc4
-loaded rules with this command pfctl -F all -f /root/pf.conf
-from pc1 tried to acces my service on pc4 and there was no problems
pc1 issued command:
#telnet 10.50.0.2 3050
Trying 10.50.0.2...

on pfsense i then dumped traffic on all interfaces
WAN interface
pfSense:~#  tcpdump -ni le0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on le0, link-type EN10MB (Ethernet), capture size 96 bytes
12:12:39.799344 IP 20.50.0.1.17797 > 10.50.0.2.3050: S 3153139180:3153139180(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2957288991 0>
12:12:42.908623 IP 20.50.0.1.17797 > 10.50.0.2.3050: S 3153139180:3153139180(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2957289003 0>
12:12:49.101611 IP 20.50.0.1.17797 > 10.50.0.2.3050: S 3153139180:3153139180(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2957289027 0>
12:12:55.073943 IP 20.50.0.1.39691 > 10.50.0.2.3050: S 121285515:121285515(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1589467175 0>
12:12:58.183807 IP 20.50.0.1.39691 > 10.50.0.2.3050: S 121285515:121285515(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1589467187 0>
12:13:04.420355 IP 20.50.0.1.39691 > 10.50.0.2.3050: S 121285515:121285515(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1589467211 0>
12:13:17.725867 IP 20.50.0.1.39691 > 10.50.0.2.3050: S 121285515:121285515(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1589467259 0>
12:13:27.849623 IP 20.50.0.1.37833 > 10.50.0.2.3050: S 1726406080:1726406080(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1700539422 0>
12:13:30.906400 IP 20.50.0.1.37833 > 10.50.0.2.3050: S 1726406080:1726406080(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1700539434 0>
12:13:37.122792 IP 20.50.0.1.37833 > 10.50.0.2.3050: S 1726406080:1726406080(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1700539458 0>
12:13:39.448390 IP 20.50.0.1.1411 > 10.50.0.2.3050: S 1221977032:1221977032(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2800433760 0>
12:13:42.544310 IP 20.50.0.1.1411 > 10.50.0.2.3050: S 1221977032:1221977032(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2800433772 0>
12:13:48.789239 IP 20.50.0.1.1411 > 10.50.0.2.3050: S 1221977032:1221977032(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2800433796 0>
12:13:56.349084 IP 20.50.0.1.43369 > 10.50.0.2.3050: S 623197096:623197096(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 498316660 0>
12:13:59.470146 IP 20.50.0.1.43369 > 10.50.0.2.3050: S 623197096:623197096(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 498316672 0>
12:14:05.642091 IP 20.50.0.1.43369 > 10.50.0.2.3050: S 623197096:623197096(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 498316696 0>
12:14:18.934221 IP 20.50.0.1.43369 > 10.50.0.2.3050: S 623197096:623197096(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 498316744 0>

LAN interface
pfSense:~#  tcpdump -ni le0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on le0, link-type EN10MB (Ethernet), capture size 96 bytes
12:12:39.799344 IP 20.50.0.1.17797 > 10.50.0.2.3050: S 3153139180:3153139180(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2957288991 0>
12:12:42.908623 IP 20.50.0.1.17797 > 10.50.0.2.3050: S 3153139180:3153139180(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2957289003 0>
12:12:49.101611 IP 20.50.0.1.17797 > 10.50.0.2.3050: S 3153139180:3153139180(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2957289027 0>
12:12:55.073943 IP 20.50.0.1.39691 > 10.50.0.2.3050: S 121285515:121285515(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1589467175 0>
12:12:58.183807 IP 20.50.0.1.39691 > 10.50.0.2.3050: S 121285515:121285515(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1589467187 0>
12:13:04.420355 IP 20.50.0.1.39691 > 10.50.0.2.3050: S 121285515:121285515(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1589467211 0>
12:13:17.725867 IP 20.50.0.1.39691 > 10.50.0.2.3050: S 121285515:121285515(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1589467259 0>
12:13:27.849623 IP 20.50.0.1.37833 > 10.50.0.2.3050: S 1726406080:1726406080(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1700539422 0>
12:13:30.906400 IP 20.50.0.1.37833 > 10.50.0.2.3050: S 1726406080:1726406080(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1700539434 0>
12:13:37.122792 IP 20.50.0.1.37833 > 10.50.0.2.3050: S 1726406080:1726406080(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 1700539458 0>
12:13:39.448390 IP 20.50.0.1.1411 > 10.50.0.2.3050: S 1221977032:1221977032(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2800433760 0>
12:13:42.544310 IP 20.50.0.1.1411 > 10.50.0.2.3050: S 1221977032:1221977032(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2800433772 0>
12:13:48.789239 IP 20.50.0.1.1411 > 10.50.0.2.3050: S 1221977032:1221977032(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 2800433796 0>
12:13:56.349084 IP 20.50.0.1.43369 > 10.50.0.2.3050: S 623197096:623197096(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 498316660 0>
12:13:59.470146 IP 20.50.0.1.43369 > 10.50.0.2.3050: S 623197096:623197096(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 498316672 0>
12:14:05.642091 IP 20.50.0.1.43369 > 10.50.0.2.3050: S 623197096:623197096(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 498316696 0>
12:14:18.934221 IP 20.50.0.1.43369 > 10.50.0.2.3050: S 623197096:623197096(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 498316744 0>

OPT1 interface
pfSense:~#  tcpdump -ni le2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on le2, link-type EN10MB (Ethernet), capture size 96 bytes
12:12:55.074689 arp who-has 20.50.0.1 tell 20.50.0.2
12:12:56.606935 arp who-has 20.50.0.1 tell 20.50.0.2
12:12:58.184179 arp who-has 20.50.0.1 tell 20.50.0.2
12:12:59.792644 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:04.420392 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:17.725956 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:19.275754 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:22.400258 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:27.849639 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:29.298290 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:30.906739 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:32.410233 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:37.122924 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:39.448504 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:41.004837 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:42.544985 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:44.186741 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:48.789311 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:56.349534 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:57.939083 arp who-has 20.50.0.1 tell 20.50.0.2
12:13:59.470493 arp who-has 20.50.0.1 tell 20.50.0.2
12:14:01.029200 arp who-has 20.50.0.1 tell 20.50.0.2
12:14:05.642239 arp who-has 20.50.0.1 tell 20.50.0.2
12:14:18.934258 arp who-has 20.50.0.1 tell 20.50.0.2
12:14:20.435657 arp who-has 20.50.0.1 tell 20.50.0.2
12:14:23.509403 arp who-has 20.50.0.1 tell 20.50.0.2

#netstat -nrf inet
Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            10.50.0.1          UGS         0    13993    le0
10.50.0.0/16       link#1             UC          0        0    le0
10.50.0.1          00:0c:29:32:bc:0c  UHLW        2       85    le0    840
20.50.0.0/16       link#3             UC          0       31    le2
20.50.0.3          00:0c:29:6c:39:c3  UHLW        1       80    le2   1166
127.0.0.1          127.0.0.1          UH          0     1699    lo0
192.168.233.0/24   link#2             UC          0        0    le1
192.168.233.1      00:50:56:c0:00:01  UHLW        1     6932    le1   1140


I tried the same config with openbsd, freebsd 6.2 and freebsd 7.0 (in place of pfsense)
Both freebsd 7.0 and pfsense 1.2.1rc4 had the same problem all other systems worked as expected.
I forgot to mention that this only happens if the IP trying to access WAN is on the same subnet as OPT1 in my case /16.


Offline ermal

  • Hero Member
  • *****
  • Posts: 3832
  • Karma: +85/-5
    • View Profile
Re: bug in pf on freebsd 7
« Reply #1 on: December 19, 2008, 08:02:14 am »
Can you draw a network diagram complete with ips and all and reformulate this?!

There is a fix for allowing that reply-to on WAN to not break certain things and it was not meant for such situations and that might break it.
The network diagram is just to see if this  is something that needs to be supported or not or it is just something that needs attention before final release.

Offline crtek

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: bug in pf on freebsd 7
« Reply #2 on: December 19, 2008, 08:17:01 am »
     PC1                                                 PC2                                                     PFSENSE                             PC3
ip 20.50.10.106/16  <------> ip2 20.50.10.105/16--|--ip1 10.50.0.1/16<----->wan ip 10.50.0.2/16--|--opt1 20.50.0.1/16 <-------> ip 20.50.0.3/16
   gw 20.50.10.105                                                 gw 10.50.0.2                       gw 10.50.0.1   |      gw 20.50.0.3                     gw 20.50.0.1
                                                                                                                                       |
                                                                                                                                       |
                                                                                                                            lan ip 192.168.233.2/24
                                                                                                                                      |
                                                                                                                                      |
                                                                                                                                      |
                                                                                                                                PC4
                                                                                                                       ip 192.168.233.1/24
                                                                                                                          gw 192.168.233.2
« Last Edit: December 19, 2008, 08:31:38 am by crtek »