Netgate SG-1000 microFirewall

Author Topic: HA Single point of failure  (Read 308 times)

0 Members and 1 Guest are viewing this topic.

Offline moelharrak

  • Newbie
  • *
  • Posts: 11
  • Karma: +0/-0
    • View Profile
HA Single point of failure
« on: December 09, 2017, 06:23:08 am »
Hi,
I'm trying to configure pfsense HA using CARP, i'm using version 2.3.4 , The Firewall is configured with DUAL WAN load-balancing ,1 LAN Interface and SYNC Interface.
The solution works fine when the master firewall is completely DOWN , But this is not a good solution. I need the backup Firewall to be the master if only one interface is DOWN in the master firwall.
any help please?

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10268
  • Karma: +1177/-313
    • View Profile
Re: HA Single point of failure
« Reply #1 on: December 09, 2017, 12:52:03 pm »
Quote
I need the backup Firewall to be the master if only one interface is DOWN
Define DOWN in that context.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline moelharrak

  • Newbie
  • *
  • Posts: 11
  • Karma: +0/-0
    • View Profile
Re: HA Single point of failure
« Reply #2 on: December 11, 2017, 11:56:02 am »
Hi,
As I mentioned I have 2 WAN and 1 LAN, I mean by DOWN if the  interface WAN1 is disconnected ( cable) or getaway for WAN1 is unreachable (monitor IP) , or in case of the LAN1 is disconnected but the WAN interfaces are working --> I need if only one of this case happen the second firewall become the master.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10268
  • Karma: +1177/-313
    • View Profile
Re: HA Single point of failure
« Reply #3 on: December 11, 2017, 02:11:06 pm »
Interface down (as in no carrier - unplugged) will trigger an HA failover.

Gateway down will trigger a multi-wan event, not an HA failover event because it is not an HA failure.

Both HA nodes should have both WANs configured identically. If a gateway is unreachable on one it should be unreachable on the other so there is no need for an HA failover.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline moelharrak

  • Newbie
  • *
  • Posts: 11
  • Karma: +0/-0
    • View Profile
Re: HA Single point of failure
« Reply #4 on: December 13, 2017, 06:18:05 am »
Hi ,
Thank you for your replay .
It's clear for the gateway not reachable ( it will be not reachable for both sides, so it's not HA failover) , however for interface down ( as in no carrier - unplugged) I tried on both side WAN and LAN but it's doesn't work , I read in a post that I need to create a group interfaces and add all interfaces to it to make HA failover work if only one interface goes down , but also no luck .

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10268
  • Karma: +1177/-313
    • View Profile
Re: HA Single point of failure
« Reply #5 on: December 13, 2017, 10:28:32 am »
It works fine. You will need to elaborate, post logs, etc.

Any down interface that has a CARP VIP will increase the advskew of all CARP VIPs on that node by net.inet.carp.ifdown_demotion_factor which is 240 by default.

That is enough to trigger all VIPs to go to BACKUP in the default configuration. (240 > 100)
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline moelharrak

  • Newbie
  • *
  • Posts: 11
  • Karma: +0/-0
    • View Profile
Re: HA Single point of failure
« Reply #6 on: December 15, 2017, 04:20:23 am »
Hi,
Still doesn't work.i didn't find net.inet.carp.ifdown_demotion_factor in System >Advanced >System Tunables .please note that I'm trying to make it works on virtual machine note real world.can be the issue ?
Attached my topology.
My configuration is :
Primary firewall :
LAN : 192.168.20.1
LAN VIPs : 192.168.20.254 VHID Group:1   Advertising frequency : 1  Skew :0
WAN1: 10.10.10.2
WAN1 VIPS : 10.10.10.4    VHID Group:2   Advertising frequency : 1   Skew :0
WAN2: 20.20.20.2
WAN1 VIPS : 20.20.20.4    VHID Group:3   Advertising frequency : 1   Skew :0
I create CARP Group in Interfaces > Interface Groups Name:CARP   Members : LAN, WAN1, WAN2

Backup Firewall
LAN : 192.168.20.2
LAN VIPs : 192.168.20.254 VHID Group:1   Advertising frequency : 1  Skew :100
WAN1: 10.10.10.3
WAN1 VIPS : 10.10.10.4    VHID Group:2   Advertising frequency : 1   Skew :100
WAN2: 20.20.20.3
WAN1 VIPS : 20.20.20.4    VHID Group:3   Advertising frequency : 1   Skew :100
I create CARP Group in Interfaces > Interface Groups Name:CARP   Members : LAN, WAN1, WAN2

when I unplug the any cable ( LAN or WAN) , only that port shows Master on backup firewall , no faillover happen.
Thank you

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10268
  • Karma: +1177/-313
    • View Profile
Re: HA Single point of failure
« Reply #7 on: December 15, 2017, 02:28:11 pm »
Yes. Being in a virtual environment might cause an unplugged cable to NOT result in an actual interface DOWN to the virtual machines because they are still connected to the vswitch. If your virtual environment supports simulating an unplugged interface there you should try that. In short, it is up to your hypervisor to actually take an interface down from the VM's perspective.

I use XenServer and that is pretty hard to simulate there - at least in the 2 minutes I devoted to trying to figure out how to do it.

You might also try just taking the interface down in software

ifconfig xn0 down


Dec 15 20:21:39    kernel       carp: 236@xn0: MASTER -> INIT (hardware interface down)
Dec 15 20:21:39    kernel       carp: demoted by 240 to 240 (interface down)
Dec 15 20:21:39    kernel       carp: 239@xn0: MASTER -> INIT (hardware interface down)
Dec 15 20:21:39    kernel       carp: demoted by 240 to 480 (interface down)
Dec 15 20:21:39    kernel       xn0: link state changed to DOWN
Dec 15 20:21:39    kernel       carp: 240@xn2: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39    kernel       ifa_maintain_loopback_route: deletion failed for interface xn2: 3
Dec 15 20:21:39    kernel       carp: 237@xn2: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39    kernel       ifa_maintain_loopback_route: deletion failed for interface xn2: 3
Dec 15 20:21:39    kernel       carp: 241@xn4: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39    kernel       ifa_maintain_loopback_route: deletion failed for interface xn4: 3
Dec 15 20:21:39    kernel       carp: 243@xn5: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39    kernel       ifa_maintain_loopback_route: deletion failed for interface xn5: 3
Dec 15 20:21:39    kernel       carp: 238@xn1: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39    kernel       ifa_maintain_loopback_route: deletion failed for interface xn1: 3
Dec 15 20:21:39    kernel       carp: 242@xn5: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39    kernel       ifa_maintain_loopback_route: deletion failed for interface xn5: 3
Dec 15 20:21:39    kernel       carp: 228@xn1: MASTER -> BACKUP (more frequent advertisement received)

Secondary takes over for all VIPS. All VIPs on primary are either INIT (the two on xn0) or BACKUP (everything else.)

ifconfig xn0 up


Dec 15 20:23:44    kernel       carp: 236@xn0: INIT -> BACKUP (initialization complete)
Dec 15 20:23:44    kernel       carp: demoted by -240 to 240 (interface up)
Dec 15 20:23:44    kernel       carp: 239@xn0: INIT -> BACKUP (initialization complete)
Dec 15 20:23:44    kernel       carp: demoted by -240 to 0 (interface up)
Dec 15 20:23:44    kernel       xn0: link state changed to UP
Dec 15 20:23:44    kernel       carp: 236@xn0: BACKUP -> INIT (hardware interface up)
Dec 15 20:23:44    kernel       carp: 236@xn0: INIT -> BACKUP (initialization complete)
Dec 15 20:23:44    kernel       carp: 239@xn0: BACKUP -> INIT (hardware interface up)
Dec 15 20:23:44    kernel       carp: 239@xn0: INIT -> BACKUP (initialization complete)
Dec 15 20:23:44    check_reload_status       Linkup starting xn0
Dec 15 20:23:44    kernel       carp: 239@xn0: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44    kernel       carp: 236@xn0: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44    kernel       carp: 241@xn4: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44    kernel       carp: 240@xn2: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44    kernel       carp: 237@xn2: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44    kernel       carp: 243@xn5: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44    kernel       carp: 242@xn5: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44    kernel       carp: 238@xn1: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44    kernel       carp: 228@xn1: BACKUP -> MASTER (preempting a slower master)

Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM