The pfSense Store

Author Topic: Trying to connect from main network into subnet.  (Read 207 times)

0 Members and 1 Guest are viewing this topic.

Offline Buayaguy

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Trying to connect from main network into subnet.
« on: December 30, 2017, 07:49:35 pm »
Hi all,

I've been setting up home routers for a long time, but it's always been simple stuff - I've never tried connecting multiple subnets before, and am setting up a little virtual lab to learn more in-depth networking topics.

Setting up pfSense was simple, and the subnet it's using (10.0.0.0/24) had zero problems immediately being able to access the Internet.

I'm trying to start simple. So I have my iMac, which is my *primary* computer - I access my virtual PCs through it as well, so pretty much everything I see on screen is coming through this computer.

You can see in the diagram below my basic setup (I'll be adding more later, but again, I just want to make sure *both* subnets can talk to each other at this point.)

Right now, I want my iMac and my Windows 10 VM (10.0.0.3/24) to be able to talk to each other. It's halfway there.

Windows 10 can successfully ping (and even log into all of) my primary router (192.168.1.1/24), my iMac (192.168.1.5/24) and my NAS (192.168.1.3/24).

The iMac can't seem to ping anything in the 10.0.0.0/24 subnet without some kind of issue.

Here's what it can do right now:

Code: [Select]
$ ping -c 4 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 56 data bytes
64 bytes from 10.0.0.1: icmp_seq=0 ttl=64 time=1.168 ms
92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 c368   0 0000  3f  01 ec92 192.168.1.5  10.0.0.1

64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.818 ms
92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 504b   0 0000  3f  01 5fb0 192.168.1.5  10.0.0.1

64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.646 ms
92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 20a0   0 0000  3f  01 8f5b 192.168.1.5  10.0.0.1

64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=0.709 ms

--- 10.0.0.1 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.646/0.835/1.168/0.202 ms

Code: [Select]
$ ping -c 4 10.0.0.3
PING 10.0.0.3 (10.0.0.3): 56 data bytes
Request timeout for icmp_seq 0
92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 3a69   0 0000  3f  01 7590 192.168.1.5  10.0.0.3

Request timeout for icmp_seq 1
92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 275a   0 0000  3f  01 889f 192.168.1.5  10.0.0.3

Request timeout for icmp_seq 2
92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 ccfb   0 0000  3f  01 e2fd 192.168.1.5  10.0.0.3


--- 10.0.0.3 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss


I've never seen a Redirect Host message EVER when I ping anything - The first example had 0% packet loss, but the second had 100% packet loss - what's going on here?



My iMac *can* access the *server* computer (192.168.1.200), since it has a direct connection back to my router (192.168.1.1) (and which I would
assume is why my VMs can access the Internet.)

But for some reason it *can't* access the WAN side of the pfSense link, even though it's on the 192.168.1.x side.

Code: [Select]
$ ping -c 4 192.168.1.201
PING 192.168.1.201 (192.168.1.201): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2

--- 192.168.1.201 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss


Thinking that maybe there was a firewall issue, I added the 2nd rule in the WAN section of pfSense, but otherwise all the WAN and LAN rules are default.

So I guess the ultimate question here is - what's stopping me from being able to get INTO 10.0.0.0 (and in this example, stopping me from at least being able to ping 10.0.0.3)?

I'm pretty much at a loss at what to do next - any help would be *extremely* appreciated.

Thanks,

       -Bryan



Edit: Should be obvious at this point, but my chart forgot to show that my pfSense WAN IP does have a /24 subnet mask (192.168.1.201/24)
« Last Edit: December 30, 2017, 07:53:24 pm by Buayaguy »

Offline tmack8080

  • Newbie
  • *
  • Posts: 3
  • Karma: +1/-0
    • View Profile
Re: Trying to connect from main network into subnet.
« Reply #1 on: December 31, 2017, 12:29:21 pm »
As far as accessing your WAN interface from the same segment, i.e., 192.168.1.0 - I suggest you open up your firewall rule just to test. See attached. I was not able to ping my WAN interface until I did that.

Offline Buayaguy

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Trying to connect from main network into subnet.
« Reply #2 on: December 31, 2017, 01:03:26 pm »
Well that was a huge help, thank you :)

I still don't want to leave it *completely* open (I know I'm still behind my main router's firewall, but this is to test how secure I can be while still having access myself.)

Opening it up worked, but then I altered it slightly, and then came up with a second rule to access the other *subnet* directly (10.0.0.0/24)

Rather than leaving the destination completely open (*), I set it to WAN Net, which still allows me to ping my 192.168.1.201 (pfSense's WAN address)

I also set up this little alias so my source is literally all 3 of my networks, so everything I have can connect through it (my 172.16.x.x needs to be reinstalled, but just knowing I can ping and *access* pfSense through web interface on 192.168.x.x was a major accomplishment for me, thank you.)

The other rule to 10.0.0.0/24 allows my iMac to ping and have web interface pfSense access on the 10.0.0.1 LAN interface as well.

The *only* thing I still can't seem to do is ping my Windows 10 VM at 10.0.0.3...

Code: [Select]
$ ping -c 4 10.0.0.3
PING 10.0.0.3 (10.0.0.3): 56 data bytes
Request timeout for icmp_seq 0
92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 3fcc   0 0000  3f  01 702d 192.168.1.5  10.0.0.3

Request timeout for icmp_seq 1
92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 2e50   0 0000  3f  01 81a9 192.168.1.5  10.0.0.3

Request timeout for icmp_seq 2
92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 3a70   0 0000  3f  01 7589 192.168.1.5  10.0.0.3


--- 10.0.0.3 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss

Once I can do that, I think I'll have all the rules in place to be able to access ANY of my other machines on 10.0.0.0/24, and use these same rules
to set up my 172.16.1.0/24 network.

Any ideas why the ping to Windows doesn't work?

Thanks again, you've been an awesome help, great birthday present for me today :)

         -Bryan
 

Offline Buayaguy

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Trying to connect from main network into subnet.
« Reply #3 on: December 31, 2017, 02:07:17 pm »
Quick update - Still not sure what's going on with the ping stuff, but I tried booting up my Linux server VM, and with the rules I put in place earlier, it looks like I *DO* have access into the 10.0.0.0 network from 192.168.1.0 -- I was able to successfully ssh into my Linux box from the iMac, so I guess those may be the only firewall rules I need.

That ping is still bugging me, although I know it's obviously not the only method of checking connectivity...

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9583
  • Karma: +1084/-309
    • View Profile
Re: Trying to connect from main network into subnet.
« Reply #4 on: December 31, 2017, 02:50:43 pm »
First, you have an asymmetric routing problem.

The default gateway for the mac is the netgear so all traffic for 10.0.0.0/24 gets sent there first. That is why you are seeing the ICMP redirects. That is lousy design.

A better way would be to make another router interface on the netgear to connect only to the pfSense WAN interface on another subnet. See the attached image. It is not identical to your situation but it demonstrates the problem.

You should probably disable outbound NAT on pfSense too. Set advanced/manual mode there then disable all of the rules.

Is the windows firewall enabled on host 10.0.0.3?
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline Buayaguy

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Trying to connect from main network into subnet.
« Reply #5 on: December 31, 2017, 03:22:05 pm »
I don't see a way to directly add another router interface, but these 2 things came to mind...

Attach #1: Maybe set each subnet as its own VLAN (based on the port it's plugged into on the Netgear)

or

Attach #2: Set up a static route, something like what I typed in here, so the Netgear can get directly to that subnet.

Would one of these fix the asymmetric problem?

And yeah, of course I forgot about the Windows firewall...turning that off allowed the ping requests through.

Would it honestly be worth leaving the local firewall on, since all my VMs are already behind pfSense, and then behind the Netgear router as well?



Thanks for the ideas so far,

       -Bryan
« Last Edit: December 31, 2017, 03:28:07 pm by Buayaguy »

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9583
  • Karma: +1084/-309
    • View Profile
Re: Trying to connect from main network into subnet.
« Reply #6 on: December 31, 2017, 04:01:14 pm »
Sorry I don't know anything about that router. I'd be running pfSense there too.

You really need to be able to route like that second design in order for this stuff to work reliably and without ICMP redirects flying all over the place.

If there's a way to coerce that router to be two inside interfaces, then you would make a static route to the 10.0.0.0/24 to that. The destination gateway would be 172.16.212.26. in the diagram I posted.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline Buayaguy

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Trying to connect from main network into subnet.
« Reply #7 on: December 31, 2017, 04:45:41 pm »
It doesn't look like it's possible to add a second interface like that...I do have that Cisco router, but we're about to start learning how to program those, and I'd rather not lose internet access while trying to figure all that out :)

I tried adding a static route from Netgear to 10.0.0.0/24, but that didn't help, so just took it back out.

I had also disabled outbound NAT like you suggested, but that killed my internal Internet access pretty much immediately, so turned it back on.

For right now, it *does* do the basics of what I need it to do - be able to communicate between all my machines in some form or other (whether just ping, or ssh...I'll be setting up a tftp server on my Linux box to send my Cisco config files to once we start doing stuff like that in class.) Yes, it still has the redirects, but class starts in like 2 days, so I really wanted to at least get the communications up and running :)

Maybe once I know more about programming Cisco devices, I'll replace my Netgear with my 1841 Router plugged into the 48 port switch and do proper VLANs for each subnet, and turn the Netgear into just an access point (because we still need wifi access in the house.)

I do have some questions about funky stuff pfSense was doing with its DHCP, but I'll post that over on the proper forum.

Thanks,

       -Bryan



Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9583
  • Karma: +1084/-309
    • View Profile
Re: Trying to connect from main network into subnet.
« Reply #8 on: December 31, 2017, 05:12:04 pm »
Yeah you could put the 1841 between both subnets but then your lab router would be in production.

Eliminating outbound NAT required a good route from the outside router to the inside addresses.

You can use it like it is if you NAT everything on the pfSense WAN, including port forwards to the inside addresses. That means you would not be able to directly access 10.0.0.3 but you could port forward, say 192.168.1.201:8443 to 10.0.0.3:8443

You can use as many inside addresses as you like so you could carve out a /26 or something and 1:1 NAT for, say, 192.168.1.192/26 to 10.0.0.192/26 and use 10.0.0.192 through 10.0.0.254 on the VMs.

You could 1:1 192.168.1.192/26 to 10.0.0.0/26 but in order to hold on to your sanity you probably want to leave the last octet the same.

In either of those cases all communication with pfSense and anything inside pfSense would be from and to 192.168.1.201 (or 192.168.1.X) and would be same-subnet to everything on 192.168.1.0/24 so the routing table would not be used.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline Buayaguy

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Trying to connect from main network into subnet.
« Reply #9 on: January 01, 2018, 12:37:18 pm »
Ok, here's the last network I'm adding (it's basically just another server + pfSense and 2 additional VMs.) (Attach 1)

Nearly every machine is talking to every other one exactly as I had hoped, with one weird issue.

As tmack8080 advised, I came up with these 2 rules so that my iMac could talk with everyone in the 10.0.0.0/24 network: (Attach 2)

So, seeing as the new network was basically the same, I did the same rules, but the 2nd rule's destination would be the local subnet (172.16.1.0/24): (Attach 3)

To avoid having to go back to previous posts, Attach 4 is the alias I made that basically says that the source location is ANY of my networks.

So what's the problem? Specifically, I can't ping either the internal LAN of the new pfSense (172.16.1.1) or any computers inside (Windows: 172.16.1.3 -- and yes, this time the Windows firewall is completely OFF.)

The first rule in my attachments allows me to access the 192.168.1.x address of either pfSense, so that one works great on both ends. The 2nd rule with the destination 10.0.0.0/24 network works *great*, both my iMac and my entire 172.16.1.0 network can access everything in the 10.0.0.0 network.

But for some reason, the 2nd rule in my 3rd attachment doesn't want to work the same way -- It won't let my iMac or my 10.0.0.0 network access *anything* in the 172.16.1.0 network. Any clues? The firewall rules are otherwise exactly the same...am I missing another setting somewhere?

Oh, and this is specifically what I'm seeing either on my iMac or my 10.0.0.0 attempts to ping:

Code: [Select]
$ ping -c 4 172.16.1.1
PING 172.16.1.1 (172.16.1.1): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2

--- 172.16.1.1 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss





Any help on this last bit would be greatly appreciated :)

        -Bryan
« Last Edit: January 01, 2018, 12:41:25 pm by Buayaguy »

Offline Buayaguy

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Trying to connect from main network into subnet.
« Reply #10 on: January 03, 2018, 12:54:49 am »
Well, I got the last bit I wanted to work - I can now get into my 172.16.1.0/24 network :)

How? I found this blog post: https://networkguy.de/?p=409

I based a static route on my Netgear router (Attach 1) on his 2nd picture with the "route -p" command listed at the bottom of the
picture, mapping his numbers to approximately what I have in place on my network.

Basically: I made a static route to the destination network (172.16.1.0/24), *through* the WAN IP of that pfSense router (192.168.1.101).

The asymmetric routing is still there, but only in specific connections:

The pfSense router (172.16.1.1) Ping Redirects the router and any computers in 10.0.0.0/24, but pings the entire 192.168.1.0/24 normally.
VMs *behind* that router ping everything normally, including the 10.0.0.0/24.

My iMac (192.168.1.5) has a Redirect Host to *both* subnets (10.0.0.0/24 and 172.16.1.0/24)

My other pfSense router (10.0.0.1) Ping Redirects anything in 172.16.1.0/24 network.
It also Ping Redirects any *computers* in 192.168.1.0/24, *BUT* it pings the router (192.168.1.1) normally.
Any machines *behind* this router ping both of the other networks (172.16.1.0/24 and 192.168.1.0/24) normally.

Again, my current router has no option for an additional interface (off the shelf model), but even with redirects, I managed to get everything to communicate,
so that's definitely something to be happy about - just in time for class to start tomorrow night as well, so I'll be able to do plenty of network testing.

Any thoughts about the weird redirects couldn't hurt - how can your router/gateway ping redirect to an entire network (first example), but all the machines behind it can ping that same network normally?

Weird.

Anyway, hope this can help someone, and thanks to everyone who helped me along to finally getting my stuff working (if not 100% cleanly.)

       -Bryan