Netgate SG-1000 microFirewall

Author Topic: Static route between 2 pfSense  (Read 1953 times)

0 Members and 1 Guest are viewing this topic.

Offline Taigar

  • Newbie
  • *
  • Posts: 13
  • Karma: +1/-0
    • View Profile
Static route between 2 pfSense
« on: March 09, 2017, 10:46:55 am »
Hello,

I've tried many things, and read a lot of topics, but I don't understand what I am doing wrong.
Probably an easy question, but I am totaly confused right now.

I have the following:


I want to be able to access the ESXi server from the workstation. What should I do? The DHCP server and gateway for the 172.25.22.0/24 network is on pfSense 1
In my understanding, I don't need anything to do on pfSense 1, because the traffic will go directly from pfsense2 to the ESXi server. Is this correct?

On the pfSense 2 I tried many different things, but it won't let me connect to the server from the workstation. On diagnostic > Ping I am able to ping the server. If I select the subnet from the workstation I am not able to ping the server. So I thought it might be a firewall problem. I created a pass all rule on the firewall, but it didn't work.

What should I do? And what can I try to troubleshoot the problem?

Offline viragomann

  • Hero Member
  • *****
  • Posts: 2821
  • Karma: +310/-1
    • View Profile
Re: Static route between 2 pfSense
« Reply #1 on: March 09, 2017, 11:15:40 am »
You need to add a route to the ESXi for the network 192.168.5.0/24 dericting to pfSense2 address 172.25.22.12.

However, if you also want to access VMs, also on these VMs a route will be needed.

Offline Taigar

  • Newbie
  • *
  • Posts: 13
  • Karma: +1/-0
    • View Profile
Re: Static route between 2 pfSense
« Reply #2 on: March 10, 2017, 02:00:33 am »
Thanks for your reply.

So I need to add a static route on the ESXi host? Is it also possible to do it without adding static routes on the ESXi host and/or VM's?

Maybe it is better to use the following setup:



If I am right, I need the following static routes:

pfSense 1:
192.168.5.0/24 gateway: 10.0.0.2

pfSense 2:
172.25.22.0/24 gateway: 10.0.0.1

Is this correct?

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10256
  • Karma: +1174/-313
    • View Profile
Re: Static route between 2 pfSense
« Reply #3 on: March 10, 2017, 02:50:21 am »
Using a transit network between routers like that is always better. :thumbsup:

Yes, your routing looks sane. Firewall rules on the transit interfaces will also need to pass the source traffic from the downstream networks.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline kpa

  • Hero Member
  • *****
  • Posts: 1267
  • Karma: +144/-6
    • View Profile
Re: Static route between 2 pfSense
« Reply #4 on: March 10, 2017, 02:54:04 am »
Yes, that's the basic pattern for any situation where you have to route between two networks that are not directly connected to the same router. Applies to site to site VPNs and other tunnel solutions as well.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10256
  • Karma: +1174/-313
    • View Profile
Re: Static route between 2 pfSense
« Reply #5 on: March 10, 2017, 02:56:54 am »
That just made me so happy to see someone come up with that. I feel real joy.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline Taigar

  • Newbie
  • *
  • Posts: 13
  • Karma: +1/-0
    • View Profile
Re: Static route between 2 pfSense
« Reply #6 on: March 10, 2017, 03:13:05 am »
I added a pass all rule on all 4 interfaces. (to test)

I am not able to ping the workstation from pfSense 1. But I am able to access the workstation from the LAN 172.25.22.0/24 net (using RDP).
I am not able to ping the ESXi server from pfSense 2, and I am not able to access the ESXi server from the workstation.

How can I troubleshoot this?

Offline Taigar

  • Newbie
  • *
  • Posts: 13
  • Karma: +1/-0
    • View Profile
Re: Static route between 2 pfSense
« Reply #7 on: March 10, 2017, 07:33:12 am »
I think this might be a NAT problem.

pfSense 1 is a fresh install, so everything is default.
pfSense 2 is our main pfSense, with a 3CX PBX behind it. So NAT is set to 'Manual Outbound NAT rule generation. (AON - Advanced Outbound NAT)'

Do I need to add some NAT mappings?

Edit: I found the problem: the ESXi host had a static IP. Apparently this isn't working. How can I make it work with a static IP?
« Last Edit: March 10, 2017, 08:15:03 am by Taigar »

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10256
  • Karma: +1174/-313
    • View Profile
Re: Static route between 2 pfSense
« Reply #8 on: March 10, 2017, 02:01:57 pm »
There should be no reason to NAT between those two networks.

Do not set gateways on the transit interfaces themselves. They should just be enabled and have IP addresses and netmasks. You create a gateway in System > Routing but do not place it on the interfaces themselves.

If there is outbound NAT for your transit interfaces, disable or delete those rules.

Not exactly sure what you're talking about with the ESXi address. Routing doesn't care how the addresses are assigned. The ESXi host needs to allow traffic from the remote subnets and have 172.25.22.1 as its default gateway. Same with any VMs you place on the 172.25.22.0/24 vSwitch.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline Taigar

  • Newbie
  • *
  • Posts: 13
  • Karma: +1/-0
    • View Profile
Re: Static route between 2 pfSense
« Reply #9 on: March 13, 2017, 03:59:51 am »
The ESXi address was set static on the host. So it was not assigned by pfSense. Once I changed it to DHCP, everything worked fine.
I will try to change it back to static (because the pfSense is a VM on this host), and see what happens

Offline MaxPF

  • Full Member
  • ***
  • Posts: 261
  • Karma: +2/-0
    • View Profile
Re: Static route between 2 pfSense
« Reply #10 on: March 23, 2017, 11:59:18 am »
There should be no reason to NAT between those two networks.

Do not set gateways on the transit interfaces themselves. They should just be enabled and have IP addresses and netmasks. You create a gateway in System > Routing but do not place it on the interfaces themselves.

If there is outbound NAT for your transit interfaces, disable or delete those rules.

Not exactly sure what you're talking about with the ESXi address. Routing doesn't care how the addresses are assigned. The ESXi host needs to allow traffic from the remote subnets and have 172.25.22.1 as its default gateway. Same with any VMs you place on the 172.25.22.0/24 vSwitch.

I have a similar setup with a transit network between two pfsense and the only way hosts from a LAN behind pfSense 1 can get to hosts on the LAN behind pfSense 2 is if I assign the gateway in the transit interfaces.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10256
  • Karma: +1174/-313
    • View Profile
Re: Static route between 2 pfSense
« Reply #11 on: March 23, 2017, 12:06:05 pm »
Then you are doing it wrong.

In the diagram above:

On pfSense 1:

Make an interface for TRANSIT with 10.0.0.1/29
Make sure firewall rules on TRANSIT pass desirable traffic from TRANSIT Net and 192.168.5.0/24
System > Routing Make a gateway TRANSIT_GW on TRANSIT for 10.0.0.2
System > Routing Make a static route for 192.168.5.0/24 to TRANSIT_GW

On pfSense 2:

Make an interface for TRANSIT with 10.0.0.2/29
Make sure firewall rules on TRANSIT pass desirable traffic from TRANSIT Net and 172.25.22.0/24
System > Routing Make a gateway TRANSIT_GW on TRANSIT for 10.0.0.1
System > Routing Make a static route for 172.25.22.0/24 to TRANSIT_GW

And you're done.

If that does not work there is, perhaps, some policy routing on the LAN interfaces that needs to be bypassed.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline MaxPF

  • Full Member
  • ***
  • Posts: 261
  • Karma: +2/-0
    • View Profile
Re: Static route between 2 pfSense
« Reply #12 on: March 23, 2017, 12:14:54 pm »
Then you are doing it wrong.

In the diagram above:

On pfSense 1:

Make an interface for TRANSIT with 10.0.0.1/29
Make sure firewall rules on TRANSIT pass desirable traffic from TRANSIT Net and 192.168.5.0/24
System > Routing Make a gateway TRANSIT_GW on TRANSIT for 10.0.0.2
System > Routing Make a static route for 192.168.5.0/24 to TRANSIT_GW

On pfSense 2:

Make an interface for TRANSIT with 10.0.0.2/29
Make sure firewall rules on TRANSIT pass desirable traffic from TRANSIT Net and 172.25.22.0/24
System > Routing Make a gateway TRANSIT_GW on TRANSIT for 10.0.0.1
System > Routing Make a static route for 172.25.22.0/24 to TRANSIT_GW

And you're done.

If that does not work there is, perhaps, some policy routing on the LAN interfaces that needs to be bypassed.

Thanks for the reply. That's exactly the way I had it, but I'll doublecheck the FW rules just in case. I'm using a /30 network for the transit network with 10.10.10.1 and 10.10.10.2 on each end, but that should not make a difference.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10256
  • Karma: +1174/-313
    • View Profile
Re: Static route between 2 pfSense
« Reply #13 on: March 23, 2017, 12:35:19 pm »
/30 is, of course, just fine.

Putting a gateway on an interface makes pfSense treat it as a WAN which is probably not what you want.

That would enable reply-to on inbound states which might mask if there was a static route in only one direction.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline MaxPF

  • Full Member
  • ***
  • Posts: 261
  • Karma: +2/-0
    • View Profile
Re: Static route between 2 pfSense
« Reply #14 on: March 23, 2017, 01:21:32 pm »
/30 is, of course, just fine.

Putting a gateway on an interface makes pfSense treat it as a WAN which is probably not what you want.

That would enable reply-to on inbound states which might mask if there was a static route in only one direction.

You were 100% right! On pfSense 2 the static route to the LAN behind pfSense 1 had a typo ::) . Fixed that, removed the gateways from both the transit interfaces' settings and everything is working.

Thanks!