pfSense Forum

pfSense English Support => NAT => Topic started by: bannerman on June 07, 2017, 11:33:57 am

Title: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: bannerman on June 07, 2017, 11:33:57 am
Hi,

I'm relatively new to pfSense and setting up a new home environment.  My goal is to send all but certain, specified traffic through an OpenVPN client connection to a remote server.  I don't want to send some streaming services through OpenVPN (like Netflix, Amazon Streaming, etc.) so I opted to not accept the routing changes coming from the VPN provider and to instead implement routing myself using firewall rules.  I've been fairly successful so far with traffic originating from my LAN side.

Where I've gotten stuck is traffic coming from the DNS resolver running on the pfSense box.  I want to make sure that all external DNS requests coming from the resolver are sent through the OpenVPN connection.  The obvious way to achieve this was to set the Outgoing Network Interface in the DNS Resolver configuration to use the interface associated with my OpenVPN connection.  However, there has been an issue at reboot, that I believe is to do with timing.  After the pfSense box is rebooted I notice that DNS traffic is not sent over the VPN.  When I look at /var/unbound/unbound.conf I see that no outgoing-interface is specified.  I think this is because the unbound service is coming up before the OpenVPN client has completed making its connection.  If I restart Unbound then traffic does start going through the VPN.

To avoid the potential for this to happen I build a floating firewall rule to send the DNS resolver traffic through the VPN.  Here is where I hit the NAT problem.  I can't figure out how to make pfSense do NAT on the DNS traffic that matches the floating firewall rule.  When I use tcpdump to view the DNS traffic I see it going out through the OpenVPN interface, but the source IP address of the traffic is my WAN IP address, not my OpenVPN IP address, and as a result I never receive a response.  Is there a way to make pfSense do NAT on traffic that matches my floating rule?

Thanks for any help on this!

BM
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: johnpoz on June 07, 2017, 12:19:32 pm
Did you actually setup a OPT interface for your client vpn connection?
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: bannerman on June 07, 2017, 12:28:38 pm
Yes, I set up an OPT interface for the OpenVPN client vpn connection.  I have an Interface Assignment that associates the ovpnc1 network port with my OPT interface.
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: johnpoz on June 07, 2017, 12:34:07 pm
I have the same sort of setup where I can route traffic out my vpn.. I have never seen the need to route dns resolver traffic out this - seems pretty pointless to me.  So you believe your isp his sniffing this traffic destined for some root or or authoritative server?  Are they intercepting all dns traffic and that is breaking your ability to resolve?

I am curious to why you would want to do this?  Other than possible getting different geographic dns responses based up the source IP of your dns query - which in the case of routing the traffic through a vpn would be the vpn end point.  This for sure could throw off your streaming, unless the vpn is in the same geographic area as you currently are.

But when I get a chance I will change my resolver to send traffic out the vpn client connection I maintain and see what happens on reboot.

Even if took a bit for the actual vpn interface to come up - resolver should still be setup to use your vpn connection, if the connection was down then it wouldn't be able to resolve - if its up then it would be able to resolve, etc.  Let me give this a test go when I get home tonight - I do need to update the snapshot anyway ;)

edit:  Ok didn't want to wait til I got home.. So I set unbound to only use my vpn interface.  So see the 209.x.x.x address that is my vps running openvpnas that I keep a tunnel connection too 24/7 so if need be I can route traffic through it.  I then rebooted (updated to current build just to kill 2 birds) and once it came back did another dnsleak test and still showing my vpn IP.  I then changed it back to my wan, and refreshed the dnsleak test page and shows my normal ISP wan IP.

So not sure what is going on with your setup - but if you created an opt interface for your vpn, does not matter the vpn interface would still be there be it the vpn is up or not.

Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: bannerman on June 07, 2017, 01:41:27 pm
Thanks for working with me on this!  The reason I'm trying to send the DNS requests through the VPN is for privacy.  My ISP isn't blocking the DNS requests, DNS resolution is certainly working fine.  I just don't want them to be able to track our activity by analyzing the DNS requests.

If I stop the OpenVPN client and restart the DNS Resolver I see that the "outgoing-interface" section is gone from the unbound configuration file.

Code: [Select]
# grep outgoing-interface /var/unbound/unbound.conf
#

If I start the OpenVPN client and then restart the DNS Resolver the outgoing-interface is put back in the config file.

Code: [Select]
# grep outgoing-interface /var/unbound/unbound.conf
outgoing-interface: 10.xxx.xxx.6
#

t may be that the presence of an interface may not be sufficient, maybe that interface needs to have an IP address?  Particularly since the config file contains the IP address of the interface, not the name of the interface.

This was why I wanted to try to address this using a firewall rule, instead of depending on the DNS Resolver to use the interface I wanted.  Since the traffic is originating from the firewall itself my understanding was that I needed a floating rule to send it through the VPN client.  When I enable the rule, it does work, however NAT is not being done on the traffic.  As a result the source IP address of my DNS requests going out on my VPN is my WAN address instead of my OpenVPN address, and I never get any responses :-).  Do the outbound NAT mappings happen before the floating firewall rules by any chance?  If so is there any way to get the firewall to NAT the traffic after the floating firewall rule?

Thanks!

BM
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: johnpoz on June 07, 2017, 02:07:16 pm
"I just don't want them to be able to track our activity by analyzing the DNS requests."

You do understand that resolver doesn't use their dns, it just sends the request over your connection to the roots and the authoritative servers directly.  So you think your ISP is actively logging all your dns requests to these servers - and then doing what with them exactly?  That seems like one tight tin foil hat..

How do you know your vpn is not logging these requests as well? ;)

"This was why I wanted to try to address this using a firewall rule, instead of depending on the DNS Resolver to use the interface I wanted."

So what interface are you going to set for the outbound interface in unbound?  the interfaces will all come up in the kernel part of the boot before any the other services start so as per my test for example vpn is going to be up way before unbound comes up.

If this is something you want to do I would look to why your vpn is not coming up?  Or takes so long to come up?  How often to do you reboot anyway?  I reboot every few weeks currently since running beta and update every so often when snaps come out.  But other than don't reboot unless there is a patch, etc.

Are you having power issues that reboot your pfsense out of the blue?  What exactly are you going to nat too if your vpn has not come up and gotten an IP?  You could always install the shellcmd package.  Have it kick off some script that restarts unbound say 5 minutes after pfsense has rebooted so your vpn interface has had time to get up, etc.

Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: bannerman on June 08, 2017, 08:18:15 am

You do understand that resolver doesn't use their dns, it just sends the request over your connection to the roots and the authoritative servers directly.  So you think your ISP is actively logging all your dns requests to these servers - and then doing what with them exactly?  That seems like one tight tin foil hat..

How do you know your vpn is not logging these requests as well? ;)

I do understand that the resolver doesn't use the ISP's DNS servers.  You're correct, the VPN provider could be logging DNS requests just as easily.  I just trust the VPN provider more that I do the ISP because of their published privacy statements and their online reputation.  Once I get a working configuration it's trivial to switch VPN providers if needed.

Quote
So what interface are you going to set for the outbound interface in unbound?  the interfaces will all come up in the kernel part of the boot before any the other services start so as per my test for example vpn is going to be up way before unbound comes up.

If I can get the firewall rules to capture the DNS traffic, then I don't really care what interface the DNS resolver uses.  i guess this is why I prefer that approach.

Quote
If this is something you want to do I would look to why your vpn is not coming up?  Or takes so long to come up?  How often to do you reboot anyway?  I reboot every few weeks currently since running beta and update every so often when snaps come out.  But other than don't reboot unless there is a patch, etc.

Are you having power issues that reboot your pfsense out of the blue?  What exactly are you going to nat too if your vpn has not come up and gotten an IP?  You could always install the shellcmd package.  Have it kick off some script that restarts unbound say 5 minutes after pfsense has rebooted so your vpn interface has had time to get up, etc.


The VPN seems to only take a couple of seconds to connect.  This doesn't look unusual to me.  I don't reboot very frequently.  This is a new box I built for pfSense, and my first time using pfSense, so I'm not sure how often it will need to be rebooted once the configuration is stabilized, but I suspect it will be very infrequent.  We almost never have power issues.  If the VPN does not come up for any reason then my goal is that there should be no internet access.  I haven't started to work on that piece yet, but DNS failing is ok with me in that case.  Thanks for the suggestion on the shellcmd package, that's a good idea.

In the meantime I started dnsmasq (using just the DNS forwarder, not the DHCP server) on an OpenWRT box that I'm using as a Wifi Access point.  I've configured the pfSense box to use the OpenWRT box as its DNS server itself and to provide it to LAN devices through DHCP.  Now all LAN-side devices are using the OpenWRT box as their DNS server, and the requests it forwards are being picked up by the pfSense LAN firewall rule and they're being sent over the OpenVPN tunnel.  It's a little clunky, but it's working for now.  I'll keep that in place until I figure out if the pfSense firewall rules can be configured to do what I want.

Thanks again for your help Johnpoz!

Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: johnpoz on June 08, 2017, 08:25:53 am
" If the VPN does not come up for any reason then my goal is that there should be no internet access. "

That is really simple - if your rules says go out vpn gateway, and gateway is not there then there is no internet access.  This is simple policy routing.  If that is your ultimate goal, then unbound not being able to resolve when the vpn doesn't come up doesn't seem like an issue at all anyway.

But to be honest, your tinfoil hat seems to be pretty freaking tight..  If you think you isp is sniffing your dns traffic ;)
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: gjaltemba on June 08, 2017, 09:11:02 am
I have LAN and VPN for Outgoing Network Interfaces and DNS Resolver starts properly after reboot. Does that work for you?
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: bannerman on June 08, 2017, 09:39:30 am
I have LAN and VPN for Outgoing Network Interfaces and DNS Resolver starts properly after reboot. Does that work for you?

If I set up LAN and VPN as the Output Network Interfaces and reboot then in /var/unbound/unbound.conf I see

Code: [Select]
outgoing-interface: 192.168.1.1

where 192.168.1.1 is my pfSense LAN IP address.

When I run tcpdump on my WAN interface I see the DNS traffic going out over the WAN interface.
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: bannerman on June 09, 2017, 03:06:01 pm
In case anyone is interested, I think I found why the DNS Resolver is coming up without an output-interface specified when the box is rebooted.

It seems that the code that writes the configuration for the unbound service is at /etc/inc/unbound.inc.  In that file there is a section that writes the output-interface configuration.  It looks like this:

Code: [Select]
// Determine interfaces to run on
$outgoingints = "";
if (!empty($unboundcfg['outgoing_interface'])) {
$outgoingints = "# Outgoing interfaces to be used\n";
$outgoing_interfaces = explode(",", $unboundcfg['outgoing_interface']);
foreach ($outgoing_interfaces as $outif) {
$outip = get_interface_ip($outif);
if (is_ipaddr($outip)) {
$outgoingints .= "outgoing-interface: $outip\n";
}
$outip = get_interface_ipv6($outif);
if (is_ipaddrv6($outip)) {
$outgoingints .= "outgoing-interface: $outip\n";
}
}
}

The code checks if each configured output interface has an IP address, and if so adds the outgoing-interface entry to the unbound config file.  If the interface doesn't have an IP address it silently skips that interface and doesn't write any outgoing-interface entry for it.  This creates a race condition on reboot where the entry will only be written only if the OpenVPN client has connected and received an IP address before the unbound configuration file is written and the unbound service started.

I modified this file slightly to check every two seconds for up to a minute for the interface to get an IP address.  I only did it for the IPv4 code, as I'm not using IPv6 yet.  This fixes the reboot issue for me.

Code: [Select]
        // Determine interfaces to run on
        $outgoingints = "";
        if (!empty($unboundcfg['outgoing_interface'])) {
                $outgoingints = "# Outgoing interfaces to be used\n";
                $outgoing_interfaces = explode(",", $unboundcfg['outgoing_interface']);
                foreach ($outgoing_interfaces as $outif) {
                        for ($outifchk=0; $outifchk < 30; $outifchk++) {
                                $outip = get_interface_ip($outif);
                                if (is_ipaddr($outip)) {
                                        $outgoingints .= "outgoing-interface: $outip\n";
                                        break;
                                }
                                sleep(2);
                        }
                        $outip = get_interface_ipv6($outif);
                        if (is_ipaddrv6($outip)) {
                                $outgoingints .= "outgoing-interface: $outip\n";
                        }
                }
        }
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: DerekChap on November 10, 2017, 02:36:55 am
Johnpoz, you must be living under a rock if you think that's tin foil hat stuff.  ISP's logging DNS actually happens all over the place - China, Russia, UK, France etc..  You look a bit silly now don't you. Ignorance is bliss though as they say  ;D
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: DerekChap on November 10, 2017, 02:39:20 am
I can verify, this issue has been around for the past 8 months or so.  For as long as I have been playing about with pfSense. Restarting the OpenVPN client even though it's already set up connection is a quick fix.  An annoying one at that.
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: Derelict on November 10, 2017, 02:40:23 am
And your VPN provider can do the same thing. You are just shifting the monitoring possibility from one party to another.
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: DerekChap on November 10, 2017, 02:44:06 am
That's true.  But when your ISP already is compelled to log, then some hope is better than none.  There are plenty of good ones out there if one does his/her research.
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: Derelict on November 10, 2017, 02:48:56 am
The bottom line here is that there are things that are pretty difficult for services running on the firewall itself.

If you want this to work 100% with pfSense in its current state, set up an inside DNS resolver that can only go out over OpenVPN (Using policy routing) and block it if it tries to go out WAN (Using mark/match and NO_WAN_EGRESS). Tell your VPN clients to use that to resolve names and not the resolver running on pfSense.

That will immediately solve your current concerns.
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: DerekChap on November 10, 2017, 02:53:07 am
Cheers for the info.  I hope though that the timing issue is looked at sometime, because regardless of the use it seems like the code could do with the tiniest bit of tweaking here.  Thanks
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: Derelict on November 10, 2017, 02:55:04 am
Guaranteed if something like that was done it would break someone else.
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: bannerman on November 10, 2017, 08:02:09 am
Guaranteed if something like that was done it would break someone else.

It would be really cool if the floating rule configuration could include a NAT configuration option too.  This would allow floating rules to be used to redirect traffic originating from the firewall through non-default outgoing interfaces and have the traffic NAT'd appropriately for that interface.
Title: Re: Sending DNS Resolver traffic through OpenVPN Client (NAT problem)
Post by: Derelict on November 10, 2017, 02:40:06 pm
It does that now.

Outbound NAT does nothing to route traffic. It merely determines what NAT occurs when traffic flows out that interface.

If the route changes, so does the NAT.

You cannot policy route traffic originating on the firewall. Period. It happens when traffic enters an interface. Traffic originating on the firewall never does that. DNS traffic from your inside DNS resolver does, thus it can be policy routed.