Netgate SG-1000 microFirewall

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Topics - muppet

Pages: [1]
1
Hi,

I'm seeing some odd behaviour with ICMP and I'm hoping someone can point me in the right direction.

My setup is as follows

ISP---[PPPoe]----WAN[pfSense]LAN---Client

My pfSense is virtualised, using VirtIO drivers, atop of KVM/Proxox.  I have turned off all checksumming.  Because of PPPoE, my MTU is 1492 for packets heading to the Internet. Traffic in general passes fine, though I have noticed some edge cases of TCP stalling/dropping which to me appears to be an MTU problem.  Thus I've gone looking and ended up here today.

From my client, a Linux PC, I see the following behaviour:
(note all pings being run with DO NOT FRAGMENT requested)

Ping with very large size and the expected reply:
Code: [Select]
tim@micro:~$ ping -M do -s 1490 8.8.8.8 -c1
PING 8.8.8.8 (8.8.8.8) 1490(1518) bytes of data.
ping: local error: Message too long, mtu=1500

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

Ping with 1444 size - I get a reply as expected.
Code: [Select]
tim@micro:~$ ping -M do -s 1444 8.8.8.8 -c1
PING 8.8.8.8 (8.8.8.8) 1444(1472) bytes of data.
1452 bytes from 8.8.8.8: icmp_seq=1 ttl=55 time=36.4 ms

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 36.456/36.456/36.456/0.000 ms

Ping with a size of 1465
Code: [Select]
tim@micro:~$ ping -M do -s 1465 8.8.8.8 -c1
PING 8.8.8.8 (8.8.8.8) 1465(1493) bytes of data.
ping: local error: Message too long, mtu=1492
^C
--- 8.8.8.8 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

Ping with a size of 1464 (one less than previous ping)
Code: [Select]
tim@micro:~$ ping -M do -s 1464 8.8.8.8 -c1
PING 8.8.8.8 (8.8.8.8) 1464(1492) bytes of data.

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

The last one there is dropped.  No "must fragment" is received.

However when I tcpdump my pppoe0 interface when sending the ping -M do -s 1464 8.8.8.8 I do see both the pings leaving the box AND the replies being received:

Code: [Select]

tim@micro:~$ ping -M do -s 1464 8.8.8.8 -c 3
PING 8.8.8.8 (8.8.8.8) 1464(1492) bytes of data.

--- 8.8.8.8 ping statistics ---

----------------------------- TCP DUMP ON PPPoE Inteface OF ABOVE 3 PINGS ---------------------------------

[2.4.2-RELEASE][admin@trogdor.muppetz.com]/root: tcpdump -i pppoe0 icmp and not host 202.56.33.251
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pppoe0, link-type NULL (BSD loopback), capture size 262144 bytes
10:27:13.670050 IP 202-137-243-17.static.nownz.co.nz > google-public-dns-a.google.com: ICMP echo request, id 21470, seq 1, length 1472
10:27:13.706961 IP google-public-dns-a.google.com > 202-137-243-17.static.nownz.co.nz: ICMP echo reply, id 21470, seq 1, length 1448
10:27:13.707021 IP google-public-dns-a.google.com > 202-137-243-17.static.nownz.co.nz: icmp
10:27:14.697492 IP 202-137-243-17.static.nownz.co.nz > google-public-dns-a.google.com: ICMP echo request, id 21470, seq 2, length 1472
10:27:14.731000 IP google-public-dns-a.google.com > 202-137-243-17.static.nownz.co.nz: ICMP echo reply, id 21470, seq 2, length 1448
10:27:14.731098 IP google-public-dns-a.google.com > 202-137-243-17.static.nownz.co.nz: icmp
10:27:15.737654 IP 202-137-243-17.static.nownz.co.nz > google-public-dns-a.google.com: ICMP echo request, id 21470, seq 3, length 1472
10:27:15.771040 IP google-public-dns-a.google.com > 202-137-243-17.static.nownz.co.nz: ICMP echo reply, id 21470, seq 3, length 1448
10:27:15.771090 IP google-public-dns-a.google.com > 202-137-243-17.static.nownz.co.nz: icmp


Doing the same command and doing a tcpdump on the LAN interface, I see only 1-way traffic:

Code: [Select]

tim@micro:~$ ping -M do -s 1464 8.8.8.8 -c 3
PING 8.8.8.8 (8.8.8.8) 1464(1492) bytes of data.

--- 8.8.8.8 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2059ms

----------------------------- TCP DUMP ON LAN Interface OF ABOVE 3 PINGS ---------------------------------

[2.4.2-RELEASE][admin@trogdor.muppetz.com]/root: tcpdump -i vtnet1 icmp and not host 202.56.33.251
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet1, link-type EN10MB (Ethernet), capture size 262144 bytes
10:28:55.129186 IP micro.muppetz.com > google-public-dns-a.google.com: ICMP echo request, id 1873, seq 1, length 1472
10:28:56.148912 IP micro.muppetz.com > google-public-dns-a.google.com: ICMP echo request, id 1873, seq 2, length 1472
10:28:57.189876 IP micro.muppetz.com > google-public-dns-a.google.com: ICMP echo request, id 1873, seq 3, length 1472


If I look at the session table for 3 failing pings I see:
Code: [Select]
LAN icmp 192.168.0.5:1874 -> 8.8.8.8:1874 0:0 3 / 0 4 KiB / 0 B
WAN icmp 202.137.243.17:32606 (192.168.0.5:1874) -> 8.8.8.8:32606 0:0 3 / 0 4 KiB / 0 B

If I look at the session table for 3 passing pings I see:

Code: [Select]
LAN icmp 192.168.0.5:1880 -> 8.8.8.8:1880 0:0 3 / 3 4 KiB / 4 KiB
WAN icmp 202.137.243.17:48100 (192.168.0.5:1880) -> 8.8.8.8:48100 0:0 3 / 3 4 KiB / 4 KiB

To recap:

ping -M do -s 1444 is the maximum I can send and get replies from my Linux PC.
ping -M do -s 1445 through -s 1464 are dropped silently somewhere in pfSense, even though I see them incoming on the PPPoE interface with TCPDump, I don't see them egress the LAN interface and the Linux PC never gets a reply.
ping -M d0 -s 1465 gives me ping: local error: Message too long, mtu=1492 as expected.

Aaahhhh I've just found they're being dropped by the Firewall!

But why?

Code: [Select]
Feb 12 10:45:03 trogdor filterlog: 9,,,1000000103,pppoe0,match,block,in,4,0x0,,56,1042,0,+,1,icmp,1468,8.8.8.8,202.137.243.17,reply,33195,11448
Feb 12 10:45:03 trogdor filterlog: 9,,,1000000103,pppoe0,match,block,in,4,0x0,,56,1042,1448,none,1,icmp,44,8.8.8.8,202.137.243.17,
Feb 12 10:45:04 trogdor filterlog: 9,,,1000000103,pppoe0,match,block,in,4,0x0,,56,1590,0,+,1,icmp,1468,8.8.8.8,202.137.243.17,reply,33195,21448
Feb 12 10:45:04 trogdor filterlog: 9,,,1000000103,pppoe0,match,block,in,4,0x0,,56,1590,1448,none,1,icmp,44,8.8.8.8,202.137.243.17,
Feb 12 10:45:05 trogdor filterlog: 9,,,1000000103,pppoe0,match,block,in,4,0x0,,56,2061,0,+,1,icmp,1468,8.8.8.8,202.137.243.17,reply,33195,31448
Feb 12 10:45:05 trogdor filterlog: 9,,,1000000103,pppoe0,match,block,in,4,0x0,,56,2061,1448,none,1,icmp,44,8.8.8.8,202.137.243.17,

What rule is causing them to be discarded?

2
Traffic Shaping / Monitoring Not Showing Queue Traffic?
« on: February 06, 2018, 01:58:58 pm »
Hi,

I've got some queues setup for my traffic.  That's all working fine, traffic goes in the right queues etc, I'm really pleased with how well it works.

This is what pftop -v queue shows me:

Code: [Select]
pfTop: Up Queue 1-6/6, View: queue, Cache: 10000                                                                                         08:53:25

QUEUE                             BW SCH  PRIO     PKTS    BYTES   DROP_P   DROP_B QLEN BORROW SUSPEN     P/S     B/S
Bulk                                 priq    0        0        0        0        0    0                     0       0
Low                                  priq        219206 25099892        0        0    0                     3     460
Medium                               priq    2   890428  102133K        0        0    0                     0       0
High                                 priq    3     2169   321266        0        0    0                   0.2       8
VeryHigh                             priq    5    12646   988551        0        0    0                     0       0
Priority                             priq    7     1269   311483        0        0    0                     0       0

However this is what the monitoring page is showing me:



There's no traffic at all, as far as it's concerned.  I have do "Use RAM Disk" ticked - could this be the problem?
Other things like traffic stats etc show as I expect.

Thanks.

3
General Questions / dPinger Query
« on: February 04, 2018, 12:32:29 pm »
Hi,

I just wondered what the expected dpinger behaviour is when it alarms, should it email me?

I have the following in my logs:

Code: [Select]
Feb  3 09:19:31 trogdor rc.gateway_alarm[96352]: >>> Gateway alarm: WAN_PPPOE (Addr:202.56.33.251 Alarm:1 RTT:106630ms RTTsd:196447ms Loss:0%)
Feb  3 09:20:16 trogdor rc.gateway_alarm[98638]: >>> Gateway alarm: WAN_PPPOE (Addr:202.56.33.251 Alarm:0 RTT:71316ms RTTsd:172082ms Loss:0%)
Feb  4 19:23:24 trogdor rc.gateway_alarm[68489]: >>> Gateway alarm: WAN_PPPOE (Addr:202.56.33.251 Alarm:1 RTT:205796ms RTTsd:618467ms Loss:0%)
Feb  4 19:24:45 trogdor rc.gateway_alarm[7300]: >>> Gateway alarm: WAN_PPPOE (Addr:202.56.33.251 Alarm:0 RTT:3883ms RTTsd:1130ms Loss:0%)

But I didn't get an email to notify me about these.

Should I have expected one, or do I misunderstand?  Should I be looking for these messages in syslog and alarming that way?

(Testing email notifications works fine, I have received them before when I forgot to configure a default Queue in the traffic shaper etc)

Thanks!

4
General Questions / See pppoe uptime from CLI?
« on: February 01, 2018, 04:35:48 pm »
Hi,

On the summary page when I login, I have the Interface tab which shows me my WAN's pppoe uptime.

I assume this is being pulled from a cli command somehow, but I can't figure out what it might be.

Can anyone share the CLI magic to display PPPOE uptime?

Thanks!

Tim

5
General Questions / Thank You!
« on: January 27, 2018, 09:14:04 pm »
I just wanted to say thankyou, to the pfSense developers and to Netgate.

I have just removed my old Cisco SRP527 box at home, which while it was still working was showing its age.

I have replaced it with one of those Chinese boxes (Yes, I know, I'm sorry, it was the best solution for me) which is running Proxmox with pfSense as one of the (4) virtual machines.  The ability to run VM's is why I didn't buy Netgate hardware, I want just one thing under the stairs.

As soon as the next paycheck arrives I will be buying Gold as a way giving back financially.

Anyway, I just wanted to say thanks.  Already I can see better performance and have a much clearer understanding at home of where my traffic is going and who is trying to connect in (more devices/systems/probes than I expected!)

This is a brilliant piece of software and I'm very grateful for the obvious time and effort that's gone into it, and this community which taught me a lot as I was waiting for my hardware to arrive.

Pages: [1]