pfSense Gold Subscription

Author Topic: Multiple Gateways on a single WAN?  (Read 291 times)

0 Members and 1 Guest are viewing this topic.

Offline Teeks

  • Newbie
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile
Multiple Gateways on a single WAN?
« on: November 21, 2017, 04:14:54 pm »
I'm not sure if this is in the right section as I only have 1 WAN port, or even if its possible with the equipment that I have here

I have a Netgate SG-2220, which has 1 WAN and 1 LAN port.  I'd like failover on the WAN port which is plugged into a switch, which has 2 connections[gateways] available.

Ive tried adding both gateways, setting gateway groups with tiers with the appropriate firewall rules, but I just cant get the second gateway to activate if the first one deactivates.

For some context.  We're in Antarctica and we use 2 satellite connections.  One connection is only available for 12 hours a day (limited satellite visibility) and the other is 24 hours, but is much slower and more expensive (we're talking 128kbps here and thousands of dollars in connection fees).  I'd like the pfsense to use the 24 hour connection (Iridium) until the 12 hour connection becomes available (BGAN) and then automatically switch over to Iridium when the BGAN connection drops later in the day.

Iridium is on 192.168.0.5 (and added as a gateway with a monitor IP of 10.20.20.30 - which is Iridium DNS)
BGAN is on 192.168.0.7 (and added as a gateway with a monitor IP of 8.8.8.8)

WAN is set to 192.168.0.8
LAN is set to 192.168.2.1 (DHCP 192.168.2.100-250)
I have 8.8.8.8 and 8.8.4.4 as DNS set in general setup, with 8.8.8.8 for the 0.5 gateway and 8.8.4.4 set for the 0.7 gateway

A gateway group exists with both gateways added, with BGAN Tier1 and Iridium Tier2

Under Interfaces > WAN > Static IPv4 config, I have IPv4 Upstream Gateway set to 192.168.0.7, otherwise the connection doesnt work.  I've tried the recommended none (On local area network interfaces the upstream gateway should be "none"), but the connection doesnt work.

For firewall rules, I've added rules and choose the group for gateway.

It all works fine if I manuually pick the gateway under Interfaces > WAN > IPv4 upstream Gateway, it's just annoying having to manually change it each time.

Any ideas?

Offline stephenw10

  • Administrator
  • Hero Member
  • *****
  • Posts: 11921
  • Karma: +468/-15
    • View Profile
Re: Multiple Gateways on a single WAN?
« Reply #1 on: November 21, 2017, 05:12:06 pm »
Antartica, nice.  ;D

You should be able to do this as long as the two WAN gateways are different so it can route packets to each independently.

It looks like you have a conflict there. You have 8.8.8.8 set as the monitor IP for BGAN but the DNS for Iridium. Each of those things sets a static route so they are in conflict. I suspect you have no DNS when on the Iridium gateway.
Swap the DNS assignments if that is the case.

By default pfSense using the DNS resolver in resolving mode with DNSSec enabled. That can only work with multiwan if you have dfault gateway switching enabled and it's better not to do that if you can.
Switch the DNS Resolver to forwarding mode and disable DNSSec. It should work with both WANs then.

You need to have a gateway set on the WAN interface to ensure outbound NAT is active in the default automatic mode. The same NAT rule will apply to both gateways here, that should be fine.

Try setting a rule to route a specific destination via a specific gateway using an IP you know responds to ping (not Google DNS though as those are already routed). If you have two rules applying to each gateway you can test connectivity to those from a client behind the firewall.

Steve

Offline GruensFroeschli

  • Little Green Frog
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5424
  • Karma: +86/-3
  • No i will not fix your computer!
    • View Profile
Re: Multiple Gateways on a single WAN?
« Reply #2 on: November 21, 2017, 05:27:58 pm »
Wow nice.

Besides the link being slow: do you have a limit on the amount of data you can transfer over each link?
How stable are the links?

You might want to increase the time of the probe interval.
No use in sending lots and lots of pings on a slow line when you know it there/not there.
We do what we must, because we can.

Asking questions the smart way: http://www.catb.org/esr/faqs/smart-questions.html

Offline Teeks

  • Newbie
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile
Re: Multiple Gateways on a single WAN?
« Reply #3 on: November 22, 2017, 11:27:18 am »
Thanks for the replies!

I've configured the DNS as instructed here, and set different IPs for monitoring, but it looks like it uses the BGAN gateway regardless (which is the default gateway).  I can disconnect the Iridium from the switch and the monitor IP still responds, so its not routing them properly.

General Settings
DNS Server settings
8.8.8.8 - 192.168.0.7
208.67.220.220 - 192.168.0.5

System/Routing/Gateways
BGAN (default) 192.168.0.7 199.193.201.12 (monitor IP)
Openport 192.168.0.5   208.67.220.220 (monitor IP)

Gateway Groups
BGAN Tier 1
Openport Tier 2

DNS Resolver
Enable DNS resolver is TICKED
DNSSEC is NOT TICKED
DNS Query Forwarding Mode TICKED

As for the links, the BGAN is on a 30gb month plan, and the Iridium on a 2gb month plan.  The BGAN gives us around 55k/sec download and an uptime of around 80%, whereas the Iridium (Openport) gives us around 8-10k/sec with a 90% uptime - not bad considering we're at the bottom of the world!

Offline Teeks

  • Newbie
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile
Re: Multiple Gateways on a single WAN?
« Reply #4 on: November 25, 2017, 10:16:46 am »
A quick update that might illicit a response

No matter how many gateways I add or which ones I choose in the firewall rules, it always uses the default gateway set under interfaces>wan. 

Any ideas on how I can specify which gateway to use under rules when I only have 1 WAN port?  It always uses the one under interfaces regardless...

Offline mvda

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: Multiple Gateways on a single WAN?
« Reply #5 on: November 27, 2017, 04:52:34 pm »
If your switch is capable of vlan tagging then you can trunk your WAN into two subinterfaces. Then you can have a separate subnet per gateway and configure the upstream gateway on the interface.
I couldn't make it work without separate vlans.

I don't really understand the idea behind upstream gateway per interface. Why should a L3 networking device with a manageable routing table have (only) one gateway per interface? 

Offline stephenw10

  • Administrator
  • Hero Member
  • *****
  • Posts: 11921
  • Karma: +468/-15
    • View Profile
Re: Multiple Gateways on a single WAN?
« Reply #6 on: November 28, 2017, 08:58:40 am »
Hmm, yes the gateways should be observed even if they're on the same interface.

The gateway monitoring to public IPs accessible via both gateways is more of an issue. You might be able to choose something internal to each providers network that can only be reached via the correct gateway but still indicates the link is up. A traceroute might show something you can use or the providers themselves might give you something. And, yes, you probably want to set the probe interval to something much longer. The default 500ms adds up to quite a lot of traffic.

Can we see your routing table?

Steve

Offline Teeks

  • Newbie
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile
Re: Multiple Gateways on a single WAN?
« Reply #7 on: November 28, 2017, 02:48:28 pm »
Thanks for help guys

Is this the routing table you need?  Or dod I need to type something at the command line?

Code: [Select]
default 192.168.0.7 UGS 909 1500 igb0
8.8.8.8 192.168.0.7 UGHS 267666 1500 igb0
127.0.0.1 link#6 UH 391288 16384 lo0
192.168.0.0/24 link#1 U 8182 1500 igb0
192.168.0.8 link#1 UHS 0 16384 lo0
192.168.2.0/24 link#2 U 3033066 1500 igb1
192.168.2.1 link#2 UHS 0 16384 lo0
199.193.201.12 192.168.0.7 UGHS 175810 1500 igb0
208.67.220.220 192.168.0.5 UGHS 413247 1500 igb0