Let me set the stage here
pfsense -- unifi switch -- att gateway
For a few months I've been troubleshooting connectivity issues with pfsense. Randomly it would drop network connectivity to the Internet and I am not able to access the firewall from the LAN. There is a thread i have out there that documented that whole fiasco but ultimately TL:DR , still don't know why its broken. I am running a 6100 and using the ix* interfaces for WAN seemed to be the culprit. When i move the WAN to the igc interfaces the random disconnects on the LAN/WAN stopped.
In between this time i moved my WAN connection to my switch so i can set up a port mirror to see whats happening on the wire. Also a cable modem swapped happen. All was well for about a month. I moved the WAN back to the ix interface and all was stable.
Today....it happened again. LAN access was not possible. Internet was very spotty. Randomly i would hit a website abut then nothing would would connect. DNS resolution to pfsense was timing out.
Access to my Unifi switch was possible and i did have a SPAN port from my ATT modem.
Packet captures show ARP requests from my firewall and the ATT gateway responding.
The ATT gateway is sending out a ARP requests and there is no ARP reply from pfsense.
Pfsense is for sure the culprit and i have pcap evidence.
I moved my SPAN port to now mirror traffic off the pfsense WAN port. The condition is the same there. ATT modem is sending a ARP requests and pfsense is not responding. pfSense is sending a ARP requests and the ATT modem is responding.
This isn't a switch issue as this was happening way before connectivity went through a switch. The switch is only used to see mirrored traffic.
@stephenw10 Finally figured out the true culprit. I still think its a NIC failure on the ix side.
I can share the pcaps if you are curious.
Has anyone seen or been through something like this on pfsense?
NIC failures cause these type of issues?
Here is a snippet of the pcap. notice the HUMAX is responding. Silicom [pfsense] never responds to HUMAXs arp..
5c02a85c-1a72-403c-b92d-b27005701706-image.png
Those pub IPs will be changed tomorrow 😊