The pfSense Store

Author Topic: High Avail. Sync broken  (Read 381 times)

0 Members and 1 Guest are viewing this topic.

Offline vigorfac

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
High Avail. Sync broken
« on: October 30, 2017, 12:58:48 pm »
Hello everyone.

I have two pfsenses so far in 2.3.
It has an interface with a lan and a sync.
I made the update in 2.4
Since the sync does not work anymore and I can not find the explanation.
Maybe you can help me.
Thank you very much.

Here are the pfsense master logs when he tries to sync.


   01/10/1930 17:45   check_reload_status      Syncing firewall
   01/10/1930 17:45   php-fpm   53724   /rc.filter_synchronize: Beginning XMLRPC sync data to https://10.88.88.2:443/xmlrpc.php.
   01/10/1930 17:45   php-fpm   53724   /rc.filter_synchronize: A communications error occurred while attempting to call XMLRPC method host_firmware_version: Unable to connect to tls://10.88.88.2:443. Error: Operation timed out
   01/10/1930 17:45   php-fpm   53724   /rc.filter_synchronize: New alert found: A communications error occurred while attempting to call XMLRPC method host_firmware_version: Unable to connect to tls://10.88.88.2:443. Error: Operation timed out
   01/10/1930 17:45   php-fpm   53724   /rc.filter_synchronize: Beginning XMLRPC sync data to https://10.88.88.2:443/xmlrpc.php.
   01/10/1930 17:46   php-fpm   53724   /rc.filter_synchronize: A communications error occurred while attempting to call XMLRPC method host_firmware_version: Unable to connect to tls://10.88.88.2:443. Error: Operation timed out
   01/10/1930 17:46   php-fpm   53724   /rc.filter_synchronize: New alert found: A communications error occurred while attempting to call XMLRPC method host_firmware_version: Unable to connect to tls://10.88.88.2:443. Error: Operation timed out
   01/10/1930 17:46   php-fpm   53724   /rc.filter_synchronize: XMLRPC versioncheck: -- 17.3
   01/10/1930 17:46   php-fpm   53724   /rc.filter_synchronize: The pfSense software configuration version of the other member could not be determined. Skipping synchronization to avoid causing a problem!

0 log on the Pfsense Slave.

Offline andipandi

  • Jr. Member
  • **
  • Posts: 55
  • Karma: +0/-1
    • View Profile
Re: High Avail. Sync broken
« Reply #1 on: October 30, 2017, 05:06:08 pm »
Same as mine? https://forum.pfsense.org/index.php?topic=139032.0

Once primary IP gets blocked, I get the timeouts too. So might be the same. No solution on that though..

Offline dotdash

  • Hero Member
  • *****
  • Posts: 1915
  • Karma: +99/-2
    • View Profile
Re: High Avail. Sync broken
« Reply #2 on: October 30, 2017, 05:54:51 pm »
Commented on the other thread, but you need to have both boxes on the same version- upgrade them both.

Offline vigorfac

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: High Avail. Sync broken
« Reply #3 on: October 31, 2017, 02:55:30 am »
Both Pfsense are on

2.4.1-RELEASE (amd64)
built on Sun Oct 22 17:26:33 CDT 2017
FreeBSD 11.1-RELEASE-p2

Offline vigorfac

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: High Avail. Sync broken
« Reply #4 on: October 31, 2017, 03:02:13 am »
@andipadi
I don't think it was the same issue, i have 0 log on secondary pfsense.

This morning I try to install from scratch on the two pfsense with 2.4.1-release version, same issue !

Offline vigorfac

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: High Avail. Sync broken
« Reply #5 on: October 31, 2017, 11:16:11 am »
I restart from scratch, sync works perfectly, then i use backup to restore my firewall, and sync broken.

Import conf brokes sync every time .....

Any idea ?

Offline vigorfac

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: High Avail. Sync broken
« Reply #6 on: November 03, 2017, 11:00:52 am »
I make a new try.

Reinstall both pfsense on 2.4

Configure Interface
Configure Sync
Configure firewall => perfectly sync on slave
configure DNS forwarder => perfectly sync on slave
Configure DHCP  => perfectly sync on slave
Configure Virtual IP => perfectly sync on slave

etc
everything work and everything is perfectly sync on slave

I make a backup of Pfsense Master and I Load this backup on Pfsense Master to check if it's work, everything is here, but syn is broken :

A communications error occurred while attempting to call XMLRPC method host_firmware_version: @ 2017-11-03 16:58:33

Somoene have an idea ?

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9059
  • Karma: +1034/-306
    • View Profile
Re: High Avail. Sync broken
« Reply #7 on: November 03, 2017, 02:08:38 pm »
What is in the system log? There should be an explicit version check line like this:

Nov 3 18:49:01    php-fpm    536    /rc.filter_synchronize: Beginning XMLRPC sync data to https://172.25.254.2:443/xmlrpc.php.
Nov 3 18:49:01    php-fpm    536    /rc.filter_synchronize: XMLRPC reload data success with https://172.25.254.2:443/xmlrpc.php (pfsense.host_firmware_version).
Nov 3 18:49:01    php-fpm    536    /rc.filter_synchronize: XMLRPC versioncheck: 17.3 -- 17.3
Nov 3 18:49:01    php-fpm    536    /rc.filter_synchronize: Beginning XMLRPC sync data to https://172.25.254.2:443/xmlrpc.php.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline vigorfac

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: High Avail. Sync broken
« Reply #8 on: November 07, 2017, 04:27:34 am »
Thank you for you reply Derelict.

I have this one on master pfsense :

   01/10/1930 17:46   php-fpm   53724   /rc.filter_synchronize: XMLRPC versioncheck: -- 17.3

both pfsense are in :

2.4.1-RELEASE (amd64)
built on Sun Oct 22 17:26:33 CDT 2017
FreeBSD 11.1-RELEASE-p2


Install with the same usb.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9059
  • Karma: +1034/-306
    • View Profile
Re: High Avail. Sync broken
« Reply #9 on: November 07, 2017, 04:48:27 am »
Sounds like your SYNC interface is not configured correctly. Can you ping across it? Check the firewall rules on it.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline vigorfac

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: High Avail. Sync broken
« Reply #10 on: November 07, 2017, 05:44:26 am »
yes master (10.88.88.1) can ping slave on 10.88.88.2
and slave can ping master on 10.88.88.1.

I have this in log now on master


Time   Process   PID   Message
Nov 7 12:40:52   php-fpm   48758   /rc.filter_synchronize: Beginning XMLRPC sync data to https://10.88.88.2:443/xmlrpc.php.
Nov 7 12:40:52   php-fpm   48758   /rc.filter_synchronize: New alert found: A communications error occurred while attempting to call XMLRPC method host_firmware_version:
Nov 7 12:40:52   php-fpm   48758   /rc.filter_synchronize: A communications error occurred while attempting to call XMLRPC method host_firmware_version:
Nov 7 12:40:43   php-fpm   38230   /rc.filter_synchronize: New alert found: A communications error occurred while attempting to call XMLRPC method filter_configure:
Nov 7 12:40:43   php-fpm   38230   /rc.filter_synchronize: A communications error occurred while attempting to call XMLRPC method filter_configure:
Nov 7 12:40:38   check_reload_status      Reloading filter
Nov 7 12:40:37   php-fpm   51322   /rc.filter_synchronize: Beginning XMLRPC sync data to https://10.88.88.2:443/xmlrpc.php.
Nov 7 12:40:36   check_reload_status      Syncing firewall
Nov 7 12:40:18   pfsense1      nginx: 2017/11/07 12:40:18 [error] 12688#100122: send() failed (54: Connection reset by peer)
Nov 7 12:40:18   php-fpm   51646   /status_logs_settings.php: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1510054818] unbound[90624:0] error: bind: address already in use [1510054818] unbound[90624:0] fatal error: could not open ports'
Nov 7 12:40:15   syslogd      kernel boot file is /boot/kernel/kernel

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9059
  • Karma: +1034/-306
    • View Profile
Re: High Avail. Sync broken
« Reply #11 on: November 07, 2017, 07:30:42 pm »
Are you passing the traffic on the sync interface on the secondary?

Are both nodes set to the same webgui settings (http/https/port) and have the same username and password set?
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline vigorfac

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: High Avail. Sync broken
« Reply #12 on: November 10, 2017, 02:59:32 am »
Sorry fo the response delay,

Yes both node are on https same port.

Both node use dedicated ionterface for the sync, no vlan .

Here are the new log :

Nov 10 09:49:29   php-fpm   34743   /rc.filter_synchronize: A communications error occurred while attempting to call XMLRPC method host_firmware_version:
Nov 10 09:49:10   check_reload_status      Reloading filter
Nov 10 09:49:10   php-fpm   57111   /rc.filter_synchronize: The pfSense software configuration version of the other member could not be determined. Skipping synchronization to avoid causing a problem!
Nov 10 09:49:10   php-fpm   57111   /rc.filter_synchronize: XMLRPC versioncheck: -- 17.3
Nov 10 09:49:10   php-fpm   57111   /rc.filter_synchronize: New alert found: A communications error occurred while attempting to call XMLRPC method host_firmware_version:
Nov 10 09:49:10   php-fpm   57111   /rc.filter_synchronize: A communications error occurred while attempting to call XMLRPC method host_firmware_version:
Nov 10 09:48:54   php-fpm   66735   /rc.filter_synchronize: Beginning XMLRPC sync data to https://10.88.88.2:443/xmlrpc.php.
Nov 10 09:48:54   php-fpm   66735   /rc.filter_synchronize: New alert found: A communications error occurred while attempting to call XMLRPC method host_firmware_version:
Nov 10 09:48:54   php-fpm   66735   /rc.filter_synchronize: A communications error occurred while attempting to call XMLRPC method host_firmware_version:
Nov 10 09:48:44   php-fpm   56798   /rc.filter_synchronize: Beginning XMLRPC sync data to https://10.88.88.2:443/xmlrpc.php.

Master and slave can ping each other.

Each time i make a change on master it's very long to validate .

Thanks for your help Derelict.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9059
  • Karma: +1034/-306
    • View Profile
Re: High Avail. Sync broken
« Reply #13 on: November 10, 2017, 03:09:15 am »
Can you bring up the webgui on the secondary at the time?

Do firewall rules pass xmlrpc (webgui) traffic on the sync interface?

If looks like the primary cannot connect to the secondary there. Need to isolate the reason why that is so.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM