pfSense Support Subscription

Author Topic: Help with Intermittent issue  (Read 206 times)

0 Members and 1 Guest are viewing this topic.

Offline almabes

  • Full Member
  • ***
  • Posts: 282
  • Karma: +28/-2
    • View Profile
Help with Intermittent issue
« on: October 12, 2017, 10:17:48 am »
Hello pfsense friends,
It's been a long time since I have posted here.  I have an issue I am dealing with on network that contains a pfsense firewall that is baffling me.  I'm not 100% sure what the root cause of the issue is, either.  Heck, it may not be the firewall. 

Hanging off the LAN (igb0) interface of the pfsense box (a SG-2440) We have Netgear L2 swtiches with 3 vlans.  VLAN 1, 2, and 3.  The switch port is configured to accept VLANS 1-3 with 2 and 3 tagged.  Physical interface igb0 also has two other VLAN interfaces configured with VLANs 2 and 3 for VoIP and HVAC control traffic.

Unpredictably, I'll get a report from my users (and subsequently monitoring software) that the "Internet has gone down".  In my investigations I have noted the following:

I cannot ping the Firewall's LAN interface during the event
I cannot ping from a device on VLAN 1 to a device on VLAN 2
I can ping other devices on VLAN 1, which should mean the switches are ok.
VoIP phones on VLAN 2 still work, and VoIP traffic still traverses the firewall to our VoIP provider.
OpenVPN never goes down.  I am able to access the configuration interface of the firewall on its LAN IP over the OpenVPN tunnel. 
I cannot access the configuration interface during an event.
I try to SSH to the firewall with puTTY, which works, and suddenly everything associated with the LAN interface comes back.  (Not sure if this is just a coincidence)

I have no schedules configured and a pretty plain vanilla rule set.  Nothing terribly wonky there.
pfSense is up-to-date. 

Any ideas? 
Can someone help me locate log messages that might provide insight on what might be going on?

Thanks in advance,
Anthony

Offline almabes

  • Full Member
  • ***
  • Posts: 282
  • Karma: +28/-2
    • View Profile
Re: Help with Intermittent issue
« Reply #1 on: October 13, 2017, 11:10:23 am »
Wow...deafening silence.

Maybe 2.4 will help. 

Offline almabes

  • Full Member
  • ***
  • Posts: 282
  • Karma: +28/-2
    • View Profile
Re: Help with Intermittent issue
« Reply #2 on: January 11, 2018, 02:01:07 pm »
This issue still occurs.
I'm a little miffed with Netgate, too, since they changed their support model. 

Anyone else out there seen anything like this?

Offline droeders

  • Newbie
  • *
  • Posts: 12
  • Karma: +2/-0
    • View Profile
Re: Help with Intermittent issue
« Reply #3 on: January 11, 2018, 09:41:55 pm »
I would start with Status -> System Logs -> System -> General.

An alternative method via SSH is:

Code: [Select]
/usr/local/sbin/clog /var/log/system.log

I'd be on the lookout for kernel messages, but other entries may be of interest as well.

Offline almabes

  • Full Member
  • ***
  • Posts: 282
  • Karma: +28/-2
    • View Profile
Re: Help with Intermittent issue
« Reply #4 on: January 12, 2018, 08:11:47 am »
Thanks for the reply.  I see nothing in the general system log that indicates any sort of a problem with igb1.

Offline mudmanc4

  • Full Member
  • ***
  • Posts: 102
  • Karma: +15/-1
  • Thou Shall Not ~kill -9 -1
    • View Profile
    • Lime/IT
Re: Help with Intermittent issue
« Reply #5 on: January 12, 2018, 09:35:55 am »
"I try to SSH to the firewall with puTTY, which works, and suddenly everything associated with the LAN interface comes back.  (Not sure if this is just a coincidence)"

Has a coincidence been ruled out by repeating this several times?

Offline almabes

  • Full Member
  • ***
  • Posts: 282
  • Karma: +28/-2
    • View Profile
Re: Help with Intermittent issue
« Reply #6 on: January 14, 2018, 12:20:10 pm »
Yes.  I ruled this out on the last time I was present.
SSH attempts didn't magically make the firewall pass traffic on the native interface again, as it appeared to have in October. 

During the last ten minute span the "event" occurred I noted this:
The switch port the firewall is plugged in to never went "down"
ARP requests to the IP of that interface came back empty when requested by a workstation on that VLAN.
I could not ping the firewall interface IP
Traffic passing over VLAN2 to the internet (same physical ingress interface) was unaffected.
Traffic passing over VLAN3 to the internet (again same physical ingress interface) was unaffected.
SSH attempts didn't "wake up" the interface (didn't expect them to but had to rule out the coincidence)

I updated to the latest firmware and rebooted.  So lets see what happens.