pfSense Support Subscription

Author Topic: CARP abnormal (?) behaviour  (Read 4633 times)

0 Members and 1 Guest are viewing this topic.

Offline Stephane

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
CARP abnormal (?) behaviour
« on: August 30, 2011, 11:55:19 am »
Hi all,
First, I want to thank all of you for PFSense. I really enjoy working on it. Actually I'm doing a lot of test trying to push this solution to my company. Although the previous version was great, we couldn't use it widely due to some limitation in the standard conf (mostly IPSEC and GRE).  With PFsense 2, I think I'll be able to replace most of our firewalls.
The problem :
using pfsense 2 RC3
My lab : 2 Redundant pfsense (A and B) using CARP, each box has 3 interfaces : WAN, SYN, LAN.
1 virtual IP for Wan.
1 virtual IP for Lan.
Pfsense A is Master
Pfsense B is Backup
What works :
If I shut down and restart pfsense A, CARP works. B becomes Master and after the reboot of A, B goes back to Backup.
If I unplug cable on A (LAN or WAN) CARP works also.
What doesn't work :
If I disable CARP on A then B becomes Master but when I re-enable CARP on A, it goes Master but B stays Master too.
The virtual ip is in fact still owned by B. The only way to fix the problem is to restart B.
I don't have this behaviour when I do the same test with pfsense 1.2.3. In that case CARP works.

Stephane



Offline ermal

  • Hero Member
  • *****
  • Posts: 3832
  • Karma: +84/-5
    • View Profile
Re: CARP abnormal (?) behaviour
« Reply #1 on: August 30, 2011, 12:29:06 pm »
Usually there should be a little delay until pfsync syncs with A.
Is this your observation or even after some time it still does not switch?

A packet trace should help on finding why this happens.

Offline Stephane

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: CARP abnormal (?) behaviour
« Reply #2 on: August 31, 2011, 08:33:20 am »
Even after few minutes it's still the same. Both A and B are Master.
 ifconfig WAN and LAN : vip is MASTER on both.

A Wan: 70.70.70.2 Lan 10.150.1.2 Syn 172.16.0.1
B Wan: 70.70.70.3 Lan 10.150.1.3 Syn 172.16.0.2
Vip Wan 70.70.70.1
Vip Lan 10.150.1.1

Packet trace on WAN
00:19:01.518690 08:00:27:59:28:30 > cc:01:0b:fc:00:00, ethertype IPv4 (0x0800), length 94: (tos 0x0, ttl 64, id 971, offset 0, flags [none], proto ICMP (1), length 80)
    70.70.70.2 > 70.70.70.5: ICMP echo request, id 25716, seq 2562, length 60
00:19:01.519043 08:00:27:59:28:30 > cc:01:0b:fc:00:00, ethertype IPv4 (0x0800), length 118: (tos 0x0, ttl 30, id 17679, offset 0, flags [none], proto GRE (47), length 104)
    70.70.70.1 > 80.80.80.2: GREv0, Flags [none], proto IPv4 (0x0800), length 84
   (tos 0x0, ttl 64, id 17679, offset 0, flags [none], proto ICMP (1), length 80)
    192.168.1.1 > 192.168.1.2: ICMP echo request, id 25716, seq 2562, length 60
00:19:01.524038 cc:01:0b:fc:00:00 > 08:00:27:59:28:30, ethertype IPv4 (0x0800), length 94: (tos 0x0, ttl 255, id 971, offset 0, flags [none], proto ICMP (1), length 80)
    70.70.70.5 > 70.70.70.2: ICMP echo reply, id 25716, seq 2562, length 60
00:19:01.763983 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0x10, ttl 255, id 41117, offset 0, flags [DF], proto VRRP (112), length 56)
    70.70.70.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 2, prio 100, authtype none, intvl 1s, length 36, addrs(7): 72.52.77.42,56.224.73.2,133.98.157.29,85.81.104.29,201.162.139.25,185.58.202.255,7.230.69.236
00:19:02.528818 08:00:27:59:28:30 > cc:01:0b:fc:00:00, ethertype IPv4 (0x0800), length 94: (tos 0x0, ttl 64, id 37026, offset 0, flags [none], proto ICMP (1), length 80)
    70.70.70.2 > 70.70.70.5: ICMP echo request, id 25716, seq 2818, length 60
00:19:02.529150 08:00:27:59:28:30 > cc:01:0b:fc:00:00, ethertype IPv4 (0x0800), length 118: (tos 0x0, ttl 30, id 38249, offset 0, flags [none], proto GRE (47), length 104)
    70.70.70.1 > 80.80.80.2: GREv0, Flags [none], proto IPv4 (0x0800), length 84
   (tos 0x0, ttl 64, id 38249, offset 0, flags [none], proto ICMP (1), length 80)
    192.168.1.1 > 192.168.1.2: ICMP echo request, id 25716, seq 2818, length 60
00:19:02.534190 cc:01:0b:fc:00:00 > 08:00:27:59:28:30, ethertype IPv4 (0x0800), length 94: (tos 0x0, ttl 255, id 37026, offset 0, flags [none], proto ICMP (1), length 80)
    70.70.70.5 > 70.70.70.2: ICMP echo reply, id 25716, seq 2818, length 60
00:19:03.174640 00:00:5e:00:01:02 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0x10, ttl 255, id 57738, offset 0, flags [DF], proto VRRP (112), length 56)
    70.70.70.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 2, prio 100, authtype none, intvl 1s, length 36, addrs(7): 72.52.77.42,56.224.73.2,133.98.157.29,85.81.104.29,201.162.139.25,185.58.202.255,7.230.69.236
00:19:03.538190 08:00:27:59:28:30 > cc:01:0b:fc:00:00, ethertype IPv4 (0x0800), length 94: (tos 0x0, ttl 64, id 10197, offset 0, flags [none], proto ICMP (1), length 80)
    70.70.70.2 > 70.70.70.5: ICMP echo request, id 25716, seq 3074, length 60
00:19:03.538332 08:00:27:59:28:30 > cc:01:0b:fc:00:00, ethertype IPv4 (0x0800), length 118: (tos 0x0, ttl 30, id 4969, offset 0, flags [none], proto GRE (47), length 104)
    70.70.70.1 > 80.80.80.2: GREv0, Flags [none], proto IPv4 (0x0800), length 84
   (tos 0x0, ttl 64, id 4969, offset 0, flags [none], proto ICMP (1), length 80)
    192.168.1.1 > 192.168.1.2: ICMP echo request, id 25716, seq 3074, length 60
00:19:03.544333 cc:01:0b:fc:00:00 > 08:00:27:59:28:30, ethertype IPv4 (0x0800), length 94: (tos 0x0, ttl 255, id 10197, offset 0, flags [none], proto ICMP (1), length 80)
    70.70.70.5 > 70.70.70.2: ICMP echo reply, id 25716, seq 3074, length 60
Packet Trace on LAN
00:20:08.737239 00:00:5e:00:01:01 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0x10, ttl 255, id 56844, offset 0, flags [DF], proto VRRP (112), length 56)
    10.150.1.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 36, addrs(7): 165.78.28.91,230.198.22.54,124.150.11.222,212.181.40.14,255.131.64.104,144.243.162.250,148.88.194.208
00:20:10.147187 00:00:5e:00:01:01 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0x10, ttl 255, id 40079, offset 0, flags [DF], proto VRRP (112), length 56)
    10.150.1.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 36, addrs(7): 165.78.28.91,230.198.22.54,124.150.11.222,212.181.40.14,255.131.64.104,144.243.162.250,148.88.194.208
00:20:11.557420 00:00:5e:00:01:01 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0x10, ttl 255, id 25843, offset 0, flags [DF], proto VRRP (112), length 56)
    10.150.1.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 36, addrs(7): 165.78.28.91,230.198.22.54,124.150.11.222,212.181.40.14,255.131.64.104,144.243.162.250,148.88.194.208
00:20:12.968668 00:00:5e:00:01:01 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0x10, ttl 255, id 23698, offset 0, flags [DF], proto VRRP (112), length 56)
    10.150.1.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 36, addrs(7): 165.78.28.91,230.198.22.54,124.150.11.222,212.181.40.14,255.131.64.104,144.243.162.250,148.88.194.208
Packet Trace on SYN
00:21:27.738473 08:00:27:f6:35:fb > 01:00:5e:00:00:f0, ethertype IPv4 (0x0800), length 582: (tos 0x10, ttl 255, id 32661, offset 0, flags [DF], proto unknown (240), length 568)
    172.16.0.2 > 224.0.0.240:  pfsync 548
00:21:28.013008 08:00:27:f6:35:fb > 01:00:5e:00:00:f0, ethertype IPv4 (0x0800), length 582: (tos 0x10, ttl 255, id 17391, offset 0, flags [DF], proto unknown (240), length 568)
    172.16.0.2 > 224.0.0.240:  pfsync 548
00:21:28.478968 08:00:27:cd:11:98 > 01:00:5e:00:00:f0, ethertype IPv4 (0x0800), length 582: (tos 0x10, ttl 255, id 25466, offset 0, flags [DF], proto unknown (240), length 568)
    172.16.0.1 > 224.0.0.240:  pfsync 548
00:21:28.944427 08:00:27:f6:35:fb > 01:00:5e:00:00:f0, ethertype IPv4 (0x0800), length 582: (tos 0x10, ttl 255, id 2825, offset 0, flags [DF], proto unknown (240), length 568)
    172.16.0.2 > 224.0.0.240:  pfsync 548
00:21:29.881374 08:00:27:cd:11:98 > 01:00:5e:00:00:f0, ethertype IPv4 (0x0800), length 494: (tos 0x10, ttl 255, id 35091, offset 0, flags [DF], proto unknown (240), length 480)
    172.16.0.1 > 224.0.0.240:  pfsync 460
00:21:29.889879 08:00:27:f6:35:fb > 01:00:5e:00:00:f0, ethertype IPv4 (0x0800), length 582: (tos 0x10, ttl 255, id 42174, offset 0, flags [DF], proto unknown (240), length 568)
    172.16.0.2 > 224.0.0.240:  pfsync 548
00:21:30.550182 08:00:27:f6:35:fb > 01:00:5e:00:00:f0, ethertype IPv4 (0x0800), length 582: (tos 0x10, ttl 255, id 47180, offset 0, flags [DF], proto unknown (240), length 568)
    172.16.0.2 > 224.0.0.240:  pfsync 548
00:21:30.881483 08:00:27:cd:11:98 > 01:00:5e:00:00:f0, ethertype IPv4 (0x0800), length 494: (tos 0x10, ttl 255, id 51093, offset 0, flags [DF], proto unknown (240), length 480)
    172.16.0.1 > 224.0.0.240:  pfsync 460
00:21:30.964528 08:00:27:f6:35:fb > 01:00:5e:00:00:f0, ethertype IPv4 (0x0800), length 582: (tos 0x10, ttl 255, id 22842, offset 0, flags [DF], proto unknown (240), length 568)
    172.16.0.2 > 224.0.0.240:  pfsync 548
00:21:31.901618 08:00:27:cd:11:98 > 01:00:5e:00:00:f0, ethertype IPv4 (0x0800), length 406: (tos 0x10, ttl 255, id 17408, offset 0, flags [DF], proto unknown (240), length 392)
    172.16.0.1 > 224.0.0.240:  pfsync 372
00:21:31.954620 08:00:27:f6:35:fb > 01:00:5e:00:00:f0, ethertype IPv4 (0x0800), length 582: (tos 0x10, ttl 255, id 18703, offset 0, flags [DF], proto unknown (240), length 568)
    172.16.0.2 > 224.0.0.240:  pfsync 548
                  
Ifconfig A
em0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
   ether 08:00:27:5b:4c:44
   media: Ethernet autoselect (1000baseT <full-duplex>)
   status: active
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
   ether 08:00:27:59:28:30
   inet 70.70.70.2 netmask 0xfffffff8 broadcast 70.70.70.7
   inet6 fe80::a00:27ff:fe59:2830%em1 prefixlen 64 scopeid 0x2
   nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
   media: Ethernet autoselect (1000baseT <full-duplex>)
   status: active
em2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
   ether 08:00:27:a2:3c:bc
   inet 10.150.1.2 netmask 0xffffff00 broadcast 10.150.1.255
   inet6 fe80::a00:27ff:fea2:3cbc%em2 prefixlen 64 scopeid 0x3
   nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
   media: Ethernet autoselect (1000baseT <full-duplex>)
   status: active
em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
   ether 08:00:27:cd:11:98
   inet 172.16.0.1 netmask 0xfffffffc broadcast 172.16.0.3
   inet6 fe80::a00:27ff:fecd:1198%em3 prefixlen 64 scopeid 0x4
   nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
   media: Ethernet autoselect (1000baseT <full-duplex>)
   status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
   options=3<RXCSUM,TXCSUM>
   inet 127.0.0.1 netmask 0xff000000
   inet6 ::1 prefixlen 128
   inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
   nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
pfsync0: flags=41<UP,RUNNING> metric 0 mtu 1460
   pfsync: syncdev: em3 syncpeer: 224.0.0.240 maxupd: 128
pflog0: flags=100<PROMISC> metric 0 mtu 33664
enc0: flags=0<> metric 0 mtu 1536
gre0: flags=9051<UP,POINTOPOINT,RUNNING,LINK0,MULTICAST> metric 0 mtu 1476
   tunnel inet 70.70.70.1 --> 80.80.80.2
   inet 192.168.1.1 --> 192.168.1.2 netmask 0xfffffffc
   inet6 fe80::a00:27ff:fe5b:4c44%gre0 prefixlen 64 scopeid 0xb
   nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
vip1: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
   inet 10.150.1.1 netmask 0xffffff00
   carp: MASTER vhid 1 advbase 1 advskew 0
vip2: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
   inet 70.70.70.1 netmask 0xffffffff
   carp: MASTER vhid 2 advbase 1 advskew 0


ifconfig B
em0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
   ether 08:00:27:f4:27:d0
   media: Ethernet autoselect (1000baseT <full-duplex>)
   status: active
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
   ether 08:00:27:10:90:b8
   inet 70.70.70.3 netmask 0xfffffff8 broadcast 70.70.70.7
   inet6 fe80::a00:27ff:fe10:90b8%em1 prefixlen 64 scopeid 0x2
   nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
   media: Ethernet autoselect (1000baseT <full-duplex>)
   status: active
em2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
   ether 08:00:27:ce:12:d8
   inet 10.150.1.3 netmask 0xffffff00 broadcast 10.150.1.255
   inet6 fe80::a00:27ff:fece:12d8%em2 prefixlen 64 scopeid 0x3
   nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
   media: Ethernet autoselect (1000baseT <full-duplex>)
   status: active
em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
   ether 08:00:27:f6:35:fb
   inet 172.16.0.2 netmask 0xfffffffc broadcast 172.16.0.3
   inet6 fe80::a00:27ff:fef6:35fb%em3 prefixlen 64 scopeid 0x4
   nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
   media: Ethernet autoselect (1000baseT <full-duplex>)
   status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
   options=3<RXCSUM,TXCSUM>
   inet 127.0.0.1 netmask 0xff000000
   inet6 ::1 prefixlen 128
   inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
   nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
pfsync0: flags=41<UP,RUNNING> metric 0 mtu 1460
   pfsync: syncdev: em3 syncpeer: 224.0.0.240 maxupd: 128
pflog0: flags=100<PROMISC> metric 0 mtu 33664
enc0: flags=0<> metric 0 mtu 1536
vip1: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
   inet 10.150.1.1 netmask 0xffffff00
   carp: MASTER vhid 1 advbase 1 advskew 100
vip2: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
   inet 70.70.70.1 netmask 0xffffffff
   carp: MASTER vhid 2 advbase 1 advskew 100
gre0: flags=9051<UP,POINTOPOINT,RUNNING,LINK0,MULTICAST> metric 0 mtu 1476
   tunnel inet 70.70.70.1 --> 80.80.80.2
   inet 192.168.1.1 --> 192.168.1.2 netmask 0xfffffffc
   inet6 fe80::a00:27ff:fef4:27d0%gre0 prefixlen 64 scopeid 0xb
   nd6 options=3<PERFORMNUD,ACCEPT_RTADV>

« Last Edit: September 01, 2011, 05:33:41 pm by Stephane »