pfSense Support Subscription

Author Topic: Problems with WAN Interface when ethernet cable is disconnected then reconnected  (Read 355 times)

0 Members and 1 Guest are viewing this topic.

Offline mhertzfeld

  • Jr. Member
  • **
  • Posts: 68
  • Karma: +4/-0
    • View Profile
I am experiencing some problems when I disconnect and reconnect the ethernet cable from my WAN interface.

The primary problem I am seeing is (what looks to me like) the WAN interface flapping when I reconnect the cable.  Other problems include the web GUI becoming very slow or not responding at all.

To get the WAN interface working again I have to reboot the box.  I've tried a few of different things like disabling\enabling the WAN interface through the GUI but the only thing that works is a reboot.

There is one error that repeats multiple times and probably tied to the root cause of the problem. 

"kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1" where 50.4.64.1 is the address same address for WAN Interface gateway before I pull the cable.

I researched the "arpresolve: can't allocate llinfo" error but haven't found a solution yet.  A lot of the posts I've found said to add "192.168.100.1" to the Reject Lease From option on the WAN Interface page, but that was when that IP address was listed in the arpresolve error.  My cable modem management IP address is "192.168.100.1" so i thought I would try that just in case.  But that did not solve the problem.  I have since removed that adress from the Reject Lease option.


Jun 8 18:12:04    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:04    kernel       em1: link state changed to UP
Jun 8 18:12:04    check_reload_status       Linkup starting em1
Jun 8 18:12:04    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:04    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:04    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:03    check_reload_status       Reloading filter
Jun 8 18:12:03    check_reload_status       Restarting OpenVPN tunnels/interfaces
Jun 8 18:12:03    check_reload_status       Restarting ipsec tunnels
Jun 8 18:12:03    check_reload_status       updating dyndns WAN_DHCP
Jun 8 18:12:03    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:03    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:03    php-fpm    20238    /rc.newwanip: Creating rrd update script
Jun 8 18:12:03    php-fpm    20238    /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Jun 8 18:12:02    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:02    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:02    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:02    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:02    check_reload_status       Reloading filter
Jun 8 18:12:02    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:02    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:12:02    kernel       em1: link state changed to DOWN
Jun 8 18:12:02    check_reload_status       Linkup starting em1
Jun 8 18:12:01    php-fpm    93063    /rc.linkup: HOTPLUG: Configuring interface wan
Jun 8 18:12:01    php-fpm    93063    /rc.linkup: DEVD Ethernet attached event for wan
Jun 8 18:12:01    check_reload_status       Reloading filter
Jun 8 18:12:00    php-fpm    20238    /rc.newwanip: Removing static route for monitor 8.8.8.8 and adding a new route through 50.4.64.1
Jun 8 18:12:00    php-fpm    20238    /rc.newwanip: ROUTING: setting default route to 50.4.64.1
Jun 8 18:12:00    php-fpm    20238    /rc.newwanip: rc.newwanip: on (IP address: 50.4.78.197) (interface: WAN[wan]) (real interface: em1).
Jun 8 18:12:00    php-fpm    20238    /rc.newwanip: rc.newwanip: Info: starting on em1.
Jun 8 18:12:00    php-fpm    66220    /rc.linkup: DEVD Ethernet detached event for wan
Jun 8 18:12:00    check_reload_status       Reloading filter
Jun 8 18:12:00    check_reload_status       updating dyndns wan
Jun 8 18:12:00    check_reload_status       Reloading filter
Jun 8 18:12:00    php-fpm    18458    /rc.start_packages: The command '/sbin/ifconfig 'em0' delete '10.10.10.1'' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'
Jun 8 18:12:00    php-fpm    18458    [pfBlockerNG] Starting cron process.
Jun 8 18:11:59    php-fpm    18458    /rc.start_packages: Restarting/Starting all packages.
Jun 8 18:11:58    check_reload_status       Restarting ipsec tunnels
Jun 8 18:11:58    php-fpm    20238    /rc.linkup: ROUTING: setting default route to 50.4.64.1
Jun 8 18:11:58    check_reload_status       rc.newwanip starting em1
Jun 8 18:11:58    check_reload_status       Starting packages
Jun 8 18:11:58    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:58    php-fpm    76105    /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 50.4.78.197 -> 50.4.78.197 - Restarting packages.
Jun 8 18:11:58    php-fpm    93063    /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP.
Jun 8 18:11:58    kernel       em1: link state changed to UP
Jun 8 18:11:58    check_reload_status       Linkup starting em1
Jun 8 18:11:58    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:57    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:57    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:57    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:57    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:57    check_reload_status       Reloading filter
Jun 8 18:11:57    check_reload_status       Restarting OpenVPN tunnels/interfaces
Jun 8 18:11:57    check_reload_status       Restarting ipsec tunnels
Jun 8 18:11:57    check_reload_status       updating dyndns WAN_DHCP
Jun 8 18:11:56    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:56    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:56    php-fpm    76105    /rc.newwanip: Creating rrd update script
Jun 8 18:11:56    php-fpm    76105    /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Jun 8 18:11:56    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:56    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:56    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:56    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:55    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:55    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:55    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:55    kernel       em1: link state changed to DOWN
Jun 8 18:11:55    check_reload_status       Linkup starting em1
Jun 8 18:11:55    php-fpm    20238    /rc.linkup: HOTPLUG: Configuring interface wan
Jun 8 18:11:55    php-fpm    20238    /rc.linkup: DEVD Ethernet attached event for wan
Jun 8 18:11:55    check_reload_status       Reloading filter
Jun 8 18:11:54    php-fpm    76105    /rc.newwanip: Removing static route for monitor 8.8.8.8 and adding a new route through 50.4.64.1
Jun 8 18:11:54    php-fpm    76105    /rc.newwanip: ROUTING: setting default route to 50.4.64.1
Jun 8 18:11:54    php-fpm    76105    /rc.newwanip: rc.newwanip: on (IP address: 50.4.78.197) (interface: WAN[wan]) (real interface: em1).
Jun 8 18:11:54    php-fpm    76105    /rc.newwanip: rc.newwanip: Info: starting on em1.
Jun 8 18:11:54    php-fpm    66220    /rc.linkup: DEVD Ethernet detached event for wan
Jun 8 18:11:54    check_reload_status       Reloading filter
Jun 8 18:11:54    check_reload_status       updating dyndns wan
Jun 8 18:11:53    check_reload_status       Reloading filter
Jun 8 18:11:53    php-fpm    48874    /rc.start_packages: The command '/sbin/ifconfig 'em0' delete '10.10.10.1'' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'
Jun 8 18:11:52    php-fpm    48874    [pfBlockerNG] Starting cron process.
Jun 8 18:11:52    php-fpm    48874    /rc.start_packages: Restarting/Starting all packages.
Jun 8 18:11:52    php-fpm    48874    /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP.
Jun 8 18:11:51    check_reload_status       Restarting ipsec tunnels
Jun 8 18:11:51    php-fpm    76105    /rc.linkup: ROUTING: setting default route to 50.4.64.1
Jun 8 18:11:51    check_reload_status       rc.newwanip starting em1
Jun 8 18:11:51    check_reload_status       Starting packages
Jun 8 18:11:51    php-fpm    48874    /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 50.4.78.197 -> 50.4.78.197 - Restarting packages.
Jun 8 18:11:51    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:51    check_reload_status       Reloading filter
Jun 8 18:11:51    check_reload_status       Restarting OpenVPN tunnels/interfaces
Jun 8 18:11:51    check_reload_status       Restarting ipsec tunnels
Jun 8 18:11:51    check_reload_status       updating dyndns WAN_DHCP
Jun 8 18:11:51    kernel       em1: link state changed to UP
Jun 8 18:11:51    check_reload_status       Linkup starting em1
Jun 8 18:11:51    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:51    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:51    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:50    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:50    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:49    php-fpm    48874    /rc.newwanip: Creating rrd update script
Jun 8 18:11:49    php-fpm    48874    /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Jun 8 18:11:49    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:49    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:49    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:49    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:49    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:49    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:49    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:49    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:48    check_reload_status       Linkup starting em1
Jun 8 18:11:48    kernel       em1: link state changed to DOWN
Jun 8 18:11:48    php-fpm    76105    /rc.linkup: HOTPLUG: Configuring interface wan
Jun 8 18:11:48    php-fpm    76105    /rc.linkup: DEVD Ethernet attached event for wan
Jun 8 18:11:48    check_reload_status       Reloading filter
Jun 8 18:11:48    php-fpm    66220    /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP.
Jun 8 18:11:47    php-fpm    48874    /rc.newwanip: Removing static route for monitor 8.8.8.8 and adding a new route through 50.4.64.1
Jun 8 18:11:47    php-fpm    48874    /rc.newwanip: ROUTING: setting default route to 50.4.64.1
Jun 8 18:11:47    php-fpm    48874    /rc.newwanip: rc.newwanip: on (IP address: 50.4.78.197) (interface: WAN[wan]) (real interface: em1).
Jun 8 18:11:47    php-fpm    48874    /rc.newwanip: rc.newwanip: Info: starting on em1.
Jun 8 18:11:47    php-fpm    37612    /rc.linkup: DEVD Ethernet detached event for wan
Jun 8 18:11:47    check_reload_status       updating dyndns wan
Jun 8 18:11:47    check_reload_status       Reloading filter
Jun 8 18:11:47    check_reload_status       Restarting OpenVPN tunnels/interfaces
Jun 8 18:11:47    check_reload_status       Restarting ipsec tunnels
Jun 8 18:11:47    check_reload_status       updating dyndns WAN_DHCP
Jun 8 18:11:45    check_reload_status       Restarting ipsec tunnels
Jun 8 18:11:45    php-fpm    48874    /rc.linkup: ROUTING: setting default route to 50.4.64.1
Jun 8 18:11:45    check_reload_status       rc.newwanip starting em1
Jun 8 18:11:45    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:45    check_reload_status       Reloading filter
Jun 8 18:11:45    php-fpm    66220    /rc.start_packages: The command '/sbin/ifconfig 'em0' delete '10.10.10.1'' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'
Jun 8 18:11:45    php-fpm    66220    [pfBlockerNG] Starting cron process.
Jun 8 18:11:45    php-fpm    66220    /rc.start_packages: Restarting/Starting all packages.
Jun 8 18:11:45    kernel       em1: link state changed to UP
Jun 8 18:11:45    check_reload_status       Linkup starting em1
Jun 8 18:11:45    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:45    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:44    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:44    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:44    check_reload_status       Starting packages
Jun 8 18:11:44    php-fpm    9542    /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 50.4.78.197 -> 50.4.78.197 - Restarting packages.
Jun 8 18:11:44    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:43    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:43    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:43    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:43    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:42    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:42    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:42    kernel       arpresolve: can't allocate llinfo for 50.4.64.1 on em1
Jun 8 18:11:42    kernel       em1: link state changed to DOWN
Jun 8 18:11:42    check_reload_status       Linkup starting em1
Jun 8 18:11:42    php-fpm    9542    /rc.newwanip: Creating rrd update script
Jun 8 18:11:42    php-fpm    9542    /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Jun 8 18:11:42    php-fpm    48874    /rc.linkup: HOTPLUG: Configuring interface wan
Jun 8 18:11:42    php-fpm    48874    /rc.linkup: DEVD Ethernet attached event for wan
Jun 8 18:11:42    check_reload_status       Reloading filter
Jun 8 18:11:41    php-fpm    37612    /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP.
Jun 8 18:11:41    php-fpm    37391    /rc.linkup: DEVD Ethernet detached event for wan
Jun 8 18:11:41    check_reload_status       Reloading filter
Jun 8 18:11:41    check_reload_status       updating dyndns wan
Jun 8 18:11:41    php-fpm    9542    /rc.newwanip: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1496959901] unbound[21478:0] error: bind: address already in use [1496959901] unbound[21478:0] fatal error: could not open ports'
Jun 8 18:11:39    php-fpm    9542    /rc.newwanip: Removing static route for monitor 8.8.8.8 and adding a new route through 50.4.64.1
Jun 8 18:11:39    php-fpm    9542    /rc.newwanip: ROUTING: setting default route to 50.4.64.1
Jun 8 18:11:39    php-fpm    9542    /rc.newwanip: rc.newwanip: on (IP address: 50.4.78.197) (interface: WAN[wan]) (real interface: em1).
Jun 8 18:11:39    php-fpm    9542    /rc.newwanip: rc.newwanip: Info: starting on em1.
Jun 8 18:11:39    check_reload_status       Reloading filter
Jun 8 18:11:39    php-fpm    48874    /rc.start_packages: The command '/sbin/ifconfig 'em0' delete '10.10.10.1'' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'
Jun 8 18:11:39    php-fpm    48874    [pfBlockerNG] Starting cron process.
Jun 8 18:11:38    php-fpm    48874    /rc.start_packages: Restarting/Starting all packages.
Jun 8 18:11:38    check_reload_status       Reloading filter

Offline mhertzfeld

  • Jr. Member
  • **
  • Posts: 68
  • Karma: +4/-0
    • View Profile
Here are some additional troubleshooting steps I've taken but have not worked for me.

Changing the monitoring ip address on my wan gateway to 8.8.8.8

Resetting all the firewall states.

Disable and Enable the WAN interface.

Offline mhertzfeld

  • Jr. Member
  • **
  • Posts: 68
  • Karma: +4/-0
    • View Profile
More info....

After I pull and reconnect the ethernet cable to the WAN port I ran a ping on the console and am seeing a very regular pattern.  I get 3-4(on average) replies followed by 3-4(on average) "ping: sendto: No route to host".  This continues until I reboot.

Offline nOw2

  • Newbie
  • *
  • Posts: 1
  • Karma: +0/-0
    • View Profile
I've had this problem for over a year and didn't find a solution until now, when I did a dive into the code and put debugging around routing commands.

I'm sharing my findings in case it helps you and others who Google this error.

Symptoms:
Reset a WAN interface and the routing dies.

In the logs:
Code: [Select]
Dec  6 06:46:35 fw kernel: arpresolve: can't allocate llinfo for 192.168.21.1 on lagg0_vlan21
In the routing table:
Code: [Select]
Destination        Gateway            Flags      Netif Expire
192.168.21.0/24    link#14            U      lagg0_vl
192.168.21.1       192.168.21.1       UGHS   lagg0_vl
192.168.21.10      link#14            UHS         lo0
Root cause:
The DNS server and gateway are the same.

Fix:
Either change DNS servers, or patch the code (pfSense 2.2 and 2.3.5):

/etc/inc/system.inc, near line 257:

Change
Code: [Select]
                            mwexec("/sbin/route {$cmd} -host {$inet6}{$dnsserver} {$gatewayip}");
To
Code: [Select]
                        if( $dnsserver != $gatewayip )
                            mwexec("/sbin/route {$cmd} -host {$inet6}{$dnsserver} {$gatewayip}");