Netgate SG-1000 microFirewall

Author Topic: DNS lookups failing periodically on VPN VLAN  (Read 341 times)

0 Members and 1 Guest are viewing this topic.

Offline toluun

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
DNS lookups failing periodically on VPN VLAN
« on: January 19, 2018, 07:11:51 am »
Every so often DNS lookups on my VPN VLAN fail. I don't really know what causes this issue, as I can't consistently repeat the error. When this error occurs I can't make DNS requests but I can ping multiple servers on the internet and the VPN WAN gateway is shown as open. When I try to look at the packets its appears the DNS requests are returning nxdomain which I think may be significant.  The only way I have found to fix the problem is to reset both the open VPN service and unbound (DNS Resolver). 

 
My VPN VLAN is setup to direct all traffic through my VPN WAN (OpenVPN).  The VPN WAN gateway is setup to monitor my VPN services DNS server and shutdown the gateway whenever pings to that server fail. The DNS Resolver is setup for the VPN network and resolving internal names. It is authoritative for my local.lan domain to preventing unneeded leakage of information.  As I believe the DNS Resolver is at fault, I have included a picture of my settings below.





Do you have any thoughts on how I could proceed in troubleshooting the problem?  I have tried using the packet manager tool to look into the packets and all I could find is what I mentioned above, that the DNS requests appeared to be returning nxdomain.  My apologies if I did not provide enough information however it is hard to replicate the error as to do so would require continuously spending time on the VPN VLAN.

Offline toluun

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: DNS lookups failing periodically on VPN VLAN
« Reply #1 on: January 21, 2018, 11:41:10 am »
Just a quick update as I have been doing some more troubleshooting.  I have narrowed down the issue being only with the dns resolver.  Fixing the problem only requires restarting unbound.  Still don't see anything in the logs that would indicate unbound failing.  I also tried adding Service Watchdog to monitor unbound and send me a notification if unbound crashes or stop working whith little success.  The issue has occurred multiple times without any notification of a restart from Service Watchdog.  Any thoughts on anything else I can try?

It may be down to creating a cron job that restarts unbound every hour or so.  Still trying to research the syntax to do that, however I would rather not take this step as it seems a workaround.

Offline toluun

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: DNS lookups failing periodically on VPN VLAN
« Reply #2 on: January 22, 2018, 10:28:58 pm »
After a couple more nights of testing I believe I have narrowed down the issue to DNSSEC.  I disabled DNSSEC and have not has any problems since.  Would like to enable DNSSEC so I will continue to look for a solution.

Offline toluun

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: DNS lookups failing periodically on VPN VLAN
« Reply #3 on: January 23, 2018, 08:29:06 pm »
Could someone maybe walk me through my resolver logs? I have to say I am thoroughly confused at some of the addresses the replies are coming from.

Code: [Select]
Jan 23 21:11:10 unbound 86566:1 info: query response REC_LAME: recursive but not authoritative server
Jan 23 21:11:10 unbound 86566:1 info: mark as REC_LAME
Jan 23 21:11:10 unbound 86566:1 info: response for data.cnn.com. A IN
Jan 23 21:11:10 unbound 86566:1 info: reply from <dsce12.akamaiedge.net.> 69.31.102.177#53
Jan 23 21:11:10 unbound 86566:1 info: query response was ANSWER
Jan 23 21:11:10 unbound 86566:1 info: resolving cnn.com. DS IN

THe first three lines repeat over and over with variation on the response address and the address the reply is from.  It seems the relevant logs data ends with:

Code: [Select]
Jan 23 21:10:15 unbound 86566:1 info: Verified that unsigned response is INSECURE
Jan 23 21:11:05 unbound 86566:1 info: resolving www.cnn.com. A IN

All these responses are from when cnn.com was actually resolved and I was able to see the page, however seeing the INSECURE has me worried. I have included similar logs for when I unbound fails to resolve a page.

Code: [Select]
Jan 23 21:35:46 unbound 86566:1 info: query response was nodata ANSWER
Jan 23 21:35:46 unbound 86566:1 info: response for n5g.akamaiedge.net. AAAA IN
Jan 23 21:35:46 unbound 86566:1 info: reply from <akamaiedge.net.> 184.26.161.192#53
Jan 23 21:35:46 unbound 86566:1 info: query response was nodata ANSWER
Jan 23 21:35:46 unbound 86566:1 info: response for n2g.akamaiedge.net. AAAA IN
Jan 23 21:35:46 unbound 86566:1 info: reply from <akamaiedge.net.> 95.101.36.192#53
Jan 23 21:35:46 unbound 86566:1 info: query response was nodata ANSWER
Jan 23 21:35:46 unbound 86566:1 info: response for www.nhl.com. A IN
Jan 23 21:35:46 unbound 86566:1 info: reply from <g.akamaiedge.net.> 88.221.81.192#53
Jan 23 21:35:46 unbound 86566:1 info: query response was ANSWER
Jan 23 21:35:46 unbound 86566:1 info: resolving nhl.com. DS IN

Once again it appears to end with

Code: [Select]
Jan 23 21:35:10 unbound 86566:1 info: NSEC3s for the referral proved no DS.
Jan 23 21:35:10 unbound 86566:1 info: Verified that unsigned response is INSECURE
Jan 23 21:35:16 unbound 86566:1 info: resolving www.nhl.com. A IN

At this point I am probably stuck with disabling DNSSEC as I probably just do not have enough knowledge to fully understand what is causing unbound to stop working (but not crashing).  I will stop spamming this thread now.
« Last Edit: January 23, 2018, 08:46:17 pm by toluun »

Offline Gertjan

  • Hero Member
  • *****
  • Posts: 2435
  • Karma: +192/-9
    • View Profile
Re: DNS lookups failing periodically on VPN VLAN
« Reply #4 on: January 24, 2018, 09:58:22 am »
At this point I am probably stuck with disabling DNSSEC as I probably just do not have enough knowledge to fully understand what is causing unbound to stop working (but not crashing).  I will stop spamming this thread now.
It's the other way around : you should keep DNSSEC enabled.
cnn.com, as many if not the most sites on the net use the classic DNS without DNSSEC - so, from a "DNSSEC-check" point of view, the answers to DNS request are unsecured. But that's ok.
Because the entire DNS, as it works for many years already, is very not secure. "DNS spoofing" really happens these days.

Btw : even forum.pfsense.org doesn't use DNSSEC  ;)

Offline toluun

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: DNS lookups failing periodically on VPN VLAN
« Reply #5 on: January 24, 2018, 10:19:25 am »
I agree, I would love to keep DNSSEC enabled but it is causing my unbound stop resolving every hour or so.  Only a manual restart of unbound fixes the issue.

Offline Johnlumsden

  • Newbie
  • *
  • Posts: 1
  • Karma: +0/-0
    • View Profile
Re: DNS lookups failing periodically on VPN VLAN
« Reply #6 on: January 27, 2018, 04:10:27 am »
While browsing the internet, you may encounter several errors and chances are you browse chrome often then the error code DNS PROBE FINISHED NXDOMAIN is likely to happen. But there’s nothing to worry about because when there is a problem, it comes with a solution.
You can find the solution here https://www.techtosh.com/how-to/error-code-dns-probe-finished-nxdomain/

Offline toluun

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: DNS lookups failing periodically on VPN VLAN
« Reply #7 on: February 01, 2018, 06:07:53 pm »
 Hmm I'm not sure this article really apples to my situation.  I'm almost positive the error is occurring at the pfsense box and not the client machines.  Mainly cause all clients are affected when the resolver fails.

Still looking for troubleahooting ideas if anyone has any.

Offline romainp

  • Full Member
  • ***
  • Posts: 139
  • Karma: +6/-0
    • View Profile
Re: DNS lookups failing periodically on VPN VLAN
« Reply #8 on: February 06, 2018, 07:09:40 pm »
Hi,
Not sure it is related to your exact issue, but I have also some really strange dns issue, explained here:

https://forum.pfsense.org/index.php?topic=143559.0

Offline toluun

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: DNS lookups failing periodically on VPN VLAN
« Reply #9 on: February 09, 2018, 12:18:14 pm »
Did your try disabling DNSSEC? Did it change anything?