Netgate Store

Author Topic: May 2nd Snapshot doesnt work, breaks everything! Beware  (Read 1447 times)

0 Members and 1 Guest are viewing this topic.

Offline LostInIgnorance

  • Sr. Member
  • ****
  • Posts: 343
  • Karma: +0/-0
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #45 on: May 15, 2018, 08:15:46 pm »
JimP, I think you're on to something with the mtu size. I can tell you that the interface (igb2) that is connecting, shows a default gateway and an IP, then it disappears from the "netstat -rnW" command screen.
I am also available after 6p CST if you would like remote access. As this appliance is a mirror of the C2758 Atom you used to sell, I am hoping there are not too many people that will experience this issue.
« Last Edit: May 15, 2018, 09:21:32 pm by LostInIgnorance »

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21839
  • Karma: +1526/-26
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #46 on: May 18, 2018, 02:15:25 pm »
The next round of snapshots should be better here. It was related to the MTU. Turns out in 11.2, FreeBSD improved dhclient so it could handle the MTU, but it took the upstream MTU unconditionally and had no way to ignore the value. In each case I've seen so far, the ISP has sent a bogus MTU back which caused two things:

1) On e1000 and some other drivers, setting the MTU causes the link to go down and back up, which triggers the interface event scripts, which restarted dhclient, which set the MTU again, which made the link go down and back up, repeat, repeat, repeat, boom.
2) On other drivers, the MTU would be set to this value but it may not have been right. In my case and for others, this was a stupid low value like 576 which meant some sites would work and others would fail or be half broken.

We have a patch in the tree now from a FreeBSD dev which will be in the next set of snapshots that lets us ignore the incoming MTU with a supersede in the dhclient config (which I also added in the tree), and hopefully all this should hopefully return sanity to cases affected by these issues.

Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline Dazog

  • Newbie
  • *
  • Posts: 19
  • Karma: +2/-0
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #47 on: May 19, 2018, 09:01:43 am »
The next round of snapshots should be better here. It was related to the MTU. Turns out in 11.2, FreeBSD improved dhclient so it could handle the MTU, but it took the upstream MTU unconditionally and had no way to ignore the value. In each case I've seen so far, the ISP has sent a bogus MTU back which caused two things:

1) On e1000 and some other drivers, setting the MTU causes the link to go down and back up, which triggers the interface event scripts, which restarted dhclient, which set the MTU again, which made the link go down and back up, repeat, repeat, repeat, boom.
2) On other drivers, the MTU would be set to this value but it may not have been right. In my case and for others, this was a stupid low value like 576 which meant some sites would work and others would fail or be half broken.

We have a patch in the tree now from a FreeBSD dev which will be in the next set of snapshots that lets us ignore the incoming MTU with a supersede in the dhclient config (which I also added in the tree), and hopefully all this should hopefully return sanity to cases affected by these issues.

Latest Build fixes issues with my DHCP WAN connection.

Bug is squashed.

Thank you for the hard work.

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21839
  • Karma: +1526/-26
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #48 on: May 19, 2018, 12:00:49 pm »
Latest Build fixes issues with my DHCP WAN connection.

Bug is squashed.

Thank you for the hard work.

Did you have the link cycling issue, the MTU issue, or both?
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline pfSenseTest

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #49 on: May 19, 2018, 01:34:04 pm »

Did you have the link cycling issue, the MTU issue, or both?

I had the link cycling issue on the Netgate MBT-4220 system and the latest snapshot fixed it.

Offline Dazog

  • Newbie
  • *
  • Posts: 19
  • Karma: +2/-0
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #50 on: May 19, 2018, 03:55:46 pm »
Latest Build fixes issues with my DHCP WAN connection.

Bug is squashed.

Thank you for the hard work.

Did you have the link cycling issue, the MTU issue, or both?

Cycling Issue.

Offline w0w

  • Sr. Member
  • ****
  • Posts: 594
  • Karma: +35/-8
  • kernel panic attack
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #51 on: Yesterday at 01:16:48 am »
I am not sure if it's related but after upgrading from 20 Apr snapshot to 19 May I lost connectivity to the internet. It is showing that PPPoE WAN is up and running, but I can not ping any IP on the internet from pfSense or LAN. I don't see anything unusual in the logs except those messages that pkg can not reach servers, rolling back ZFS snapshot restores connection immediately.

P.S. Looks like in some stage it got connected because it shows my dynamic DNS as updated and once it reinstalled packages, but can not get package list anymore, ping 8.8.8.8 100% lost, traceroute does not even start to trace.

What can I do else to analyze it?
« Last Edit: Yesterday at 02:09:49 am by w0w »

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21839
  • Karma: +1526/-26
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #52 on: Yesterday at 07:27:51 am »
I am not sure if it's related but after upgrading from 20 Apr snapshot to 19 May I lost connectivity to the internet. It is showing that PPPoE WAN is up and running, but I can not ping any IP on the internet from pfSense or LAN. I don't see anything unusual in the logs except those messages that pkg can not reach servers, rolling back ZFS snapshot restores connection immediately.

P.S. Looks like in some stage it got connected because it shows my dynamic DNS as updated and once it reinstalled packages, but can not get package list anymore, ping 8.8.8.8 100% lost, traceroute does not even start to trace.

What can I do else to analyze it?

That doesn't quite sound like it's related to anything in this thread. Start a new thread and post details of your setup there, at least show the routing table and what the gateways list/page looks like, maybe the config.xml entries for the gateways you have configured. You can redact any IP addresses in that info.
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline w0w

  • Sr. Member
  • ****
  • Posts: 594
  • Karma: +35/-8
  • kernel panic attack
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #53 on: Yesterday at 12:29:39 pm »
OK jimp.

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21839
  • Karma: +1526/-26
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #54 on: Today at 08:54:25 am »
Looks like we can consider all of these issues resolved as far as I can see. Every system I'm aware of that has tried the updated code is working properly now.
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline tmushy

  • Newbie
  • *
  • Posts: 10
  • Karma: +0/-0
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #55 on: Today at 09:33:56 am »
I can confirm the latest snapshot has indeed fixed all my issues!
Thank you for resolving this. Working great now

Offline LostInIgnorance

  • Sr. Member
  • ****
  • Posts: 343
  • Karma: +0/-0
    • View Profile
Re: May 2nd Snapshot doesnt work, breaks everything! Beware
« Reply #56 on: Today at 09:41:47 am »
JimP, I am sorry I didn't get to it earlier, but I was out of town. I just upgraded this morning and everything is working correctly as it should. Thanks for getting this fixed.