pfSense Gold Subscription

Author Topic: gateway tier priority backwards?  (Read 355 times)

0 Members and 1 Guest are viewing this topic.

Offline repomanz

  • Jr. Member
  • **
  • Posts: 38
  • Karma: +0/-0
    • View Profile
gateway tier priority backwards?
« on: December 30, 2017, 08:18:40 pm »
Hi folks - maybe i'm reading this wrong in the help pages.  What I understand is that in a gateway group, the interface connection has priority values.  Meaning, if i have interfaces that are defined as tier 1 within the gateway group that they should take priority over interfaces within the same gateway group that are defined as tier 2.

Not sure why but my tier 1 interfaces seem to be taking lower priority than the tier 2.  Meaning, the amount of traffic coming into and out of my tier 2 defined interfaces is much greater than the tier 1 interfaces.  I have checked logs and such and my tier 1 interfaces are not going down. 

thoughts?

repo

Offline repomanz

  • Jr. Member
  • **
  • Posts: 38
  • Karma: +0/-0
    • View Profile
Re: gateway tier priority backwards?
« Reply #1 on: January 01, 2018, 12:57:07 pm »
** bump  **

Do i understand the documentation correctly? Is this a bug perhaps?  My tier interfaces take priority over the tier interfaces.  A side note, my interfaces are openvpn clients.

Offline heper

  • Hero Member
  • *****
  • Posts: 2696
  • Karma: +255/-11
    • View Profile
Re: gateway tier priority backwards?
« Reply #2 on: January 01, 2018, 05:30:47 pm »
your vpn provider probably pushes a default route that overwrites the policy routing.

check 'dont pull routes' & try again

Offline repomanz

  • Jr. Member
  • **
  • Posts: 38
  • Karma: +0/-0
    • View Profile
Re: gateway tier priority backwards?
« Reply #3 on: January 01, 2018, 05:33:07 pm »
I have checked on all vpn sessions (for both tier 1 and tier 2 providers)

- don't pull routes
- don't add/remove routes

Offline repomanz

  • Jr. Member
  • **
  • Posts: 38
  • Karma: +0/-0
    • View Profile
Re: gateway tier priority backwards?
« Reply #4 on: January 02, 2018, 10:35:30 pm »
Hi folks - here is what i'm talking about for example.  1 day of traffic. Notice provider B appears to be handling all the traffic.

VPN Provider A Connection # 1 UDP4   up   Tue Jan 2 0:16:44 2018   xxx.xxx.xxx.xxx:16234   xxx.xxx.xxx.xxx   xxx.xxx.xxx.xxx:1197   14.62 MiB / 629 KiB   
   
VPN Provider A Connection # 2 UDP4   up   Tue Jan 2 0:16:39 2018   xxx.xxx.xxx.xxx:36865   xxx.xxx.xxx.xxx   xxx.xxx.xxx.xxx:1197   16.01 MiB / 4.67 MiB   
   
VPN Provider A Connection # 3 UDP4   up   Tue Jan 2 0:16:39 2018   xxx.xxx.xxx.xxx:48592   xxx.xxx.xxx.xxx   xxx.xxx.xxx.xxx:1197   2.22 GiB / 95.97 MiB   
   
VPN Provider A Connection # 4 UDP4   up   Tue Jan 2 0:16:40 2018   xxx.xxx.xxx.xxx:58877   xxx.xxx.xxx.xxx   xxx.xxx.xxx.xxx:1197   16.01 MiB / 4.63 MiB   
   
VPN Provider A Connection # 5 UDP4   up   Tue Jan 2 0:16:40 2018   xxx.xxx.xxx.xxx:8290   xxx.xxx.xxx.xxx   xxx.xxx.xxx.xxx:1197   16.02 MiB / 4.62 MiB   
   
VPN Provider B Connection # 1 UDP4   up   Tue Jan 2 14:53:30 2018   xxx.xxx.xxx.xxx:24262   xxx.xxx.xxx.xxx   xxx.xxx.xxx.xxx:1197   601.69 MiB / 1.14 GiB   
   
VPN Provider B Connection # 2 UDP4   up   Tue Jan 2 14:53:29 2018   xxx.xxx.xxx.xxx:19959   xxx.xxx.xxx.xxx   xxx.xxx.xxx.xxx:1197   4.01 GiB / 3.48 GiB   
   
VPN Provider B Connection # 3 UDP4   up   Tue Jan 2 14:53:25 2018   xxx.xxx.xxx.xxx:49613   xxx.xxx.xxx.xxx   xxx.xxx.xxx.xxx:1197   99.69 MiB / 1.44 GiB   
   
VPN Provider B Connection # 4 UDP4   up   Tue Jan 2 14:53:33 2018   xxx.xxx.xxx.xxx:13192   xxx.xxx.xxx.xxx   xxx.xxx.xxx.xxx:1197   181.82 MiB / 1.54 GiB   
   
VPN Provider B Connection # 5 UDP4   up   Tue Jan 2 14:53:32 2018   xxx.xxx.xxx.xxx:51944   xxx.xxx.xxx.xxx   xxx.xxx.xxx.xxx:1197   1.54 GiB / 2.48 GiB   

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9606
  • Karma: +1090/-309
    • View Profile
Re: gateway tier priority backwards?
« Reply #5 on: January 02, 2018, 10:51:42 pm »
What is defined on your gateway groups? WAN interfaces or VPN provider gateways?

Post your policy routing rules and gateway group configurations.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline repomanz

  • Jr. Member
  • **
  • Posts: 38
  • Karma: +0/-0
    • View Profile
Re: gateway tier priority backwards?
« Reply #6 on: January 02, 2018, 10:55:39 pm »
Each OpenVPN Client has it's own interface
Each of these interfaces, including WAN are in the single gateway group

Gateway group configuration:
WAN = Never
All VPN interfaces = Tier 1
Trigger = packet loss or high latency

LAN rule routes traffic out of gateway group.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9606
  • Karma: +1090/-309
    • View Profile
Re: gateway tier priority backwards?
« Reply #7 on: January 03, 2018, 11:56:11 am »
Load balancing has no way to know how much traffic a particular state will end up transferring when it is created. It balances states among connections, not traffic. The fact that you show approximately the same number of states on each connection means it is working.

ETA: Moving to Multi-WAN
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline repomanz

  • Jr. Member
  • **
  • Posts: 38
  • Karma: +0/-0
    • View Profile
Re: gateway tier priority backwards?
« Reply #8 on: January 03, 2018, 12:08:58 pm »
I'm not following your statement.  If you look at the 2 providers, VPN A shows megabytes. VPN B shows gigabytes.  And if i understand, the LB method is round robin so over time these numbers should be roughly the same.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9606
  • Karma: +1090/-309
    • View Profile
Re: gateway tier priority backwards?
« Reply #9 on: January 03, 2018, 12:12:42 pm »
It doesn't matter. They both have 5 states. Again, when you make a connection it has no idea how much traffic is going to go over it.

If VPN A has 5 states established and VPN B has 4, the new connection goes out VPN B. How much traffic has gone over the states in the past is not evaluated. In fact, the one with the most traffic on it might be idle at the time the new state is created and never transfer another byte.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline repomanz

  • Jr. Member
  • **
  • Posts: 38
  • Karma: +0/-0
    • View Profile
Re: gateway tier priority backwards?
« Reply #10 on: January 03, 2018, 12:17:08 pm »
are there any plans to change LB algorithms? like least connection?

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9606
  • Karma: +1090/-309
    • View Profile
Re: gateway tier priority backwards?
« Reply #11 on: January 03, 2018, 12:23:30 pm »
No. I made a reference already that shows why that would be folly. A state that has transferred 100GB might never transfer another byte. There is no way for the firewall to know or predict with any accuracy what is going to happen based on what has happened. And it is impossible to move states between interfaces to correct later.

You can skew the algorithm regarding the number of states put on each circuit with the gateway weights but that is about it.

For example two gateways in a group, both tier 1. The first gateway has a weight of 4, the second a weight of 1. 4 out of 5 states - or 80% - will be created on the first gateway, 20% on the second.

Another reference:

https://forum.pfsense.org/index.php?topic=124373.msg697215#msg697215
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline repomanz

  • Jr. Member
  • **
  • Posts: 38
  • Karma: +0/-0
    • View Profile
Re: gateway tier priority backwards?
« Reply #12 on: January 03, 2018, 02:44:48 pm »
I did an experiment.  I turned off all but 1 of the VPN interfaces with VPN B.  That left me with

5 VPN A clients running
1 VPN B client running

The gategroup detected the 4 VPN B clients were down and now all traffic is being routed through the sole VPN B client.

The only difference between the two VPN client configurations is that VPN B uses a TLS key where as VPN A does not.

This looks like a defect. I have 30ish clients running various activities behind this firewall.  I should see a significant increase in the VPN A clients activity but I am not.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9606
  • Karma: +1090/-309
    • View Profile
Re: gateway tier priority backwards?
« Reply #13 on: January 03, 2018, 03:05:46 pm »
Post your rules.

There is not a problem with Load Balancing. It does what it does very well. See the other thread I posted. Every time I test it because someone claims it doesn't work right it works fine. Not going to do it again.

Quote
5 VPN A clients running
1 VPN B client running
I don't understand at all what you are doing there. Going to need a much better description.

If you have 5 VPN clients running to one provider they all need assigned interfaces and they all need gateways in the gateway group if you want them to be utilized in that manner.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline repomanz

  • Jr. Member
  • **
  • Posts: 38
  • Karma: +0/-0
    • View Profile
Re: gateway tier priority backwards?
« Reply #14 on: January 03, 2018, 03:16:30 pm »
I've pseudo posted my rules above.  What do you want to see specifically? glad to grab screen shots of the areas you want to look at.