pfSense Gold Subscription

Author Topic: 2.4.1: local DNS not working  (Read 1183 times)

0 Members and 1 Guest are viewing this topic.

Offline JKnott

  • Hero Member
  • *****
  • Posts: 895
  • Karma: +29/-4
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #30 on: November 06, 2017, 08:59:39 am »
Quote
Repeat - and force to use IPv4 and IPv6 :
dig -4 @2607:fea8:4cdf:abcd:216:17ff:fea7:xyz google.com A
and
dig -6 @2607:fea8:4cdf:abcd:216:17ff:fea7:xyz google.com A

$ dig -4 @2607:fea8:4cdf:abcd:216:17ff:fea7:xyz google.com A
dig: couldn't get address for '2607:fea8:4cdf:ef00:216:17ff:fea7:f2d3': address family not supported

As expected, forcing an IPv4 query to an IPv6 address won't work.

$ dig -6 @2607:fea8:4cdf:abcd:216:17ff:fea7:xyz google.com A

; <<>> DiG 9.9.9-P1 <<>> -6 @2607:fea8:4cdf:ef00:216:17ff:fea7:f2d3 google.com A
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

Same result as before.

Quote
Are you passing IPv6 DNS into that interface?

Are you listening for DNS on that interface? Meaning does the resolver have that interface or All interfaces selected?

My desktop computer is configured only with IPv6 DNS addresses, so yes IPv6 is enabled.  Also, as mentioned earlier, forwarder works fine, so that would rule out any address issues.  I did not make any changes from 2.4.0, where resolver worked to 2.4.1, where it fails.

root: netstat -an | grep LISTEN | grep 53
tcp4       0      0 127.0.0.1.953          *.*                    LISTEN
tcp6       0      0 ::1.53                 *.*                    LISTEN
tcp4       0      0 127.0.0.1.53           *.*                    LISTEN

It's listening only on the loopback.  Prior to the update, I had resolver network interfaces configured to all local networks and outgoing interface to WAN.  However, after the update, that failed, giving errors as described in the other thread.

However, I think I just found the problem.  When I was trying to resolve the problem initially, someone mentioned to select All & All for the interfaces.  That didn't work and the config wouldn't let me save just the local networks.  I'd get the error "This system is configured to use the DNS Resolver as its DNS server, so Localhost or All must be selected in Network Interfaces".  I had been running with WAN for the outgoing interface and localhost for network, as that was one of the 2 allowed in the configuration.  I had previously tried ALL and it failed, but appears to be working now.  Why is it now necessary to choose ALL, when previously it worked with selected interfaces?

Offline hda

  • Sr. Member
  • ****
  • Posts: 588
  • Karma: +30/-3
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #31 on: November 06, 2017, 09:54:09 am »
Why is it now necessary to choose ALL...
Well, it isn't mandatory. See my post #29.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9047
  • Karma: +1032/-306
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #32 on: November 06, 2017, 12:17:41 pm »
Quote
It is listening only on loopback.
Then you need to port forward DNS requests to the loopback interface or make it listen on the interfaces you are trying to use as name servers by selecting those interfaces in the resolver configuration.

Like I said MANY TIMES! Select "All" for the interfaces in the resolver config and you'll get this:


tcp4       0      0 *.53                   *.*                    LISTEN
tcp6       0      0 *.53                   *.*                    LISTEN


Who knows what weird configuration you had that was working and now isn't.

This is all I see for unbound from 2.4.0 to 2.4.1.

https://redmine.pfsense.org/issues/7884

https://redmine.pfsense.org/issues/7814

« Last Edit: November 06, 2017, 12:23:25 pm by Derelict »
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline JKnott

  • Hero Member
  • *****
  • Posts: 895
  • Karma: +29/-4
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #33 on: November 06, 2017, 02:00:41 pm »
Why is it now necessary to choose ALL...
Well, it isn't mandatory. See my post #29.

I had something similar to that and it work well, until 2.4.1.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9047
  • Karma: +1032/-306
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #34 on: November 06, 2017, 02:08:25 pm »
Sigh.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 14276
  • Karma: +1329/-191
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #35 on: November 07, 2017, 05:10:55 am »
My main computer uses static configuration for DNS, with IPv6 addresses for pfSense and Google DNS servers.  Devices that connect via DHCP get the IPv4 address for pfSense DNS for the 1st DNS server and 8.8.8.8 & 4.4.4.4 for 2nd & 3rd.
  This is so freaking BORKED! 

Derelict is right on the money when he says that is broken..

When you state " first DNS to try and Google as the 2nd, should the first fail."

Doesn't work that way!!  Your logic is that client always asks pfsense... What does pfsense return for what you queried? NX, Refused, Error?  Timeout.. What?  So then you believe client asks google for whatever it is you asked.  Now the next thing it wants to lookup up you believe it goes back to ask your 1st listed dns?

Sorry but it does NOT work that way!!  You have no control over which dns is going to get asked what from a client..  Especially with windows...   When you point a client to more than 1 nameserver.. These multiple name server need to be able to return the same info... You do not point a ns that resolve local, and then also point to a ns that can not resolve your local..  If for some reason your client asks google for a local record.. its going to send back NX... Once a dns client gets NX why would it go ask a different NS.. It was told that record doesn't exist.. Not that ns timed out.. Or sorry go ask someone else I don't know.. It got told that record does not exist - period.. So why should ask some other NS hoping the answer is different..

> dig @8.8.8.8 pfsense.local.lan

; <<>> DiG 9.11.2 <<>> @8.8.8.8 pfsense.local.lan
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 52361
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1


My client will not ask another NS in its list after getting that.. There is no point to it..  So pointing clients to different ns that can not resolve the same info is asking for problems plain and simple..

From MS
https://technet.microsoft.com/en-us/library/cc961411.aspx
"If at any point the DNS Client service receives a negative response from a server, it removes every server on that adapter from consideration during this search. For example, if in step 2, the first server on Alternate Adapter A gave a negative response, the DNS Client service would not send the query to any other server on the list for Alternate Adapter A."

« Last Edit: November 07, 2017, 05:17:58 am by johnpoz »
- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- Please don't PM me for personal help
- if you want to say thanks applaud or https://www.freebsdfoundation.org/donate/
1x SG-2440 2.3.4_p1 (work)
1x 2.4.1-RELEASE on VM esxi 6.5 (home)

Offline JKnott

  • Hero Member
  • *****
  • Posts: 895
  • Karma: +29/-4
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #36 on: November 07, 2017, 06:09:17 am »
My main computer uses static configuration for DNS, with IPv6 addresses for pfSense and Google DNS servers.  Devices that connect via DHCP get the IPv4 address for pfSense DNS for the 1st DNS server and 8.8.8.8 & 4.4.4.4 for 2nd & 3rd.
  This is so freaking BORKED! 

Derelict is right on the money when he says that is broken..

When you state " first DNS to try and Google as the 2nd, should the first fail."

Doesn't work that way!!  Your logic is that client always asks pfsense... What does pfsense return for what you queried? NX, Refused, Error?  Timeout.. What?  So then you believe client asks google for whatever it is you asked.  Now the next thing it wants to lookup up you believe it goes back to ask your 1st listed dns?

Sorry but it does NOT work that way!!  You have no control over which dns is going to get asked what from a client..  Especially with windows...   When you point a client to more than 1 nameserver.. These multiple name server need to be able to return the same info... You do not point a ns that resolve local, and then also point to a ns that can not resolve your local..  If for some reason your client asks google for a local record.. its going to send back NX... Once a dns client gets NX why would it go ask a different NS.. It was told that record doesn't exist.. Not that ns timed out.. Or sorry go ask someone else I don't know.. It got told that record does not exist - period.. So why should ask some other NS hoping the answer is different..

> dig @8.8.8.8 pfsense.local.lan

; <<>> DiG 9.11.2 <<>> @8.8.8.8 pfsense.local.lan
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 52361
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1


My client will not ask another NS in its list after getting that.. There is no point to it..  So pointing clients to different ns that can not resolve the same info is asking for problems plain and simple..

From MS
https://technet.microsoft.com/en-us/library/cc961411.aspx
"If at any point the DNS Client service receives a negative response from a server, it removes every server on that adapter from consideration during this search. For example, if in step 2, the first server on Alternate Adapter A gave a negative response, the DNS Client service would not send the query to any other server on the list for Alternate Adapter A."

Who said anything about running Windows?  I try to avoid using it.  My computers run mainly Linux and I provided a link to a Linux man page about resolv.conf that showed it worked exactly as I said.  Also, what's borked about a device that uses DHCP to get it's IPv4 config also using IPv4 DNS addresses?  Last I checked IPv4 DHCP can't provide IPv6 addresses and I don't run DHCPv6.  So, what exactly is "borked"?

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 14276
  • Karma: +1329/-191
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #37 on: November 07, 2017, 02:32:41 pm »
Pointing to local NS and public NS that can not resolve your local is BORKED - plain and simple.. This has been this way since the start of DNS..

Please show me your sniff of your dns queries coming from your machine showing it asking another NS after you got a NX from some public NS..

Unless the 1st server ask returns SERVFAIL or Timesouts - why should the clients dns move to the second NS in the list.. NX stops all queries.. Since it got an answer that the record in question does not exist, why bother asking anything else.

And even using say dnsmasq as local dns client that sends to all NS in parallel, it uses the first one to answer.. And doesn't look at any other responses, etc..

So how exactly is your linux box setup.. And what did it query and what did it get in response... Saying that you have pfsense, google for your NS 1, 2, 3 doesn't tell us what the problem is or isn't.

Do a simple query from the client.. Does it resolve or not.. If you query pfsense unbound on whatever address its listening on.. What is the response... This is a simple dig or nslookup or host command.  Please post the output on why you think dns is not working...   Saying its broke is just nonsense..

You don't tell you mech the car is broke.. You tell him or show him what is not working.. Unless you show us what is not working.. I resolve in this order staying in the dns theme - unless I can duplicate the error its PEBKAC..  Unless you can show me what its going on or not happening, PEBKAC..

Once I get back PEBKAC, I don't bother looking for other answers until actually get shown some info to work with..

How exactly do you expect unbound to work with

Quote
Outgoing Network Interfaces:
Localhost

Come on dude - really!!
« Last Edit: November 07, 2017, 02:39:31 pm by johnpoz »
- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- Please don't PM me for personal help
- if you want to say thanks applaud or https://www.freebsdfoundation.org/donate/
1x SG-2440 2.3.4_p1 (work)
1x 2.4.1-RELEASE on VM esxi 6.5 (home)

Offline kncar77

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #38 on: November 08, 2017, 11:58:26 am »
I concur. My DNS resolver broke after updating to 2.4.1
Have been using the same setup and have run every update since 2.2 and never had any problem.
I'm using a VPN client and am forcing the DNS resolver to query the root requests through the VPN tunnel rather than WAN, otherwise my middle eastern ISP will hijack the requests (verified with DNS leaktest). It has worked like a charm until 30 minutes ago after updating to 2.4.1.
When I manually add a DNS server (8.8.8.8 for example) on the clients it works but using the pfsense DNS resolver, then it times out.

All solved. Turned out to be a freak issue with my VPN provider rather than pfSense
« Last Edit: November 12, 2017, 02:19:09 am by kncar77 »

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9047
  • Karma: +1032/-306
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #39 on: November 08, 2017, 12:01:22 pm »
Please enumerate, completely and thoroughly, the steps you have taken in the DNS resolver, The OpenVPN client connection, and policy routing to effect such a configuration.

Saying "resolver broke" doesn't give anyone anything to go on.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline JKnott

  • Hero Member
  • *****
  • Posts: 895
  • Karma: +29/-4
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #40 on: November 08, 2017, 04:49:33 pm »
Pointing to local NS and public NS that can not resolve your local is BORKED - plain and simple.. This has been this way since the start of DNS..

Please show me your sniff of your dns queries coming from your machine showing it asking another NS after you got a NX from some public NS..

Unless the 1st server ask returns SERVFAIL or Timesouts - why should the clients dns move to the second NS in the list.. NX stops all queries.. Since it got an answer that the record in question does not exist, why bother asking anything else.

And even using say dnsmasq as local dns client that sends to all NS in parallel, it uses the first one to answer.. And doesn't look at any other responses, etc..

So how exactly is your linux box setup.. And what did it query and what did it get in response... Saying that you have pfsense, google for your NS 1, 2, 3 doesn't tell us what the problem is or isn't.

Do a simple query from the client.. Does it resolve or not.. If you query pfsense unbound on whatever address its listening on.. What is the response... This is a simple dig or nslookup or host command.  Please post the output on why you think dns is not working...   Saying its broke is just nonsense..

You don't tell you mech the car is broke.. You tell him or show him what is not working.. Unless you show us what is not working.. I resolve in this order staying in the dns theme - unless I can duplicate the error its PEBKAC..  Unless you can show me what its going on or not happening, PEBKAC..

Once I get back PEBKAC, I don't bother looking for other answers until actually get shown some info to work with..

How exactly do you expect unbound to work with

Quote
Outgoing Network Interfaces:
Localhost

Come on dude - really!!

Actually, my local IPv6 addresses are available via public DNS.  I use a DNS service to do that.  So, having an external DNS is just a back up.  I have attached a capture showing the fall back in action.  I modified the local DNS address in resolv.conf, to force queries to the Google DNS.  Take a look at the transactions for google.ca.  You'll first see an attempt to the phony local address.  It fails, so the Google DNS is tried next.  There are 3 separate examples in the capture for A, AAAA and MX records.  In each case, the local lookup fails and then Google is tried.  This is exactly the behavior I said would happen and also as described in the resolv.conf man page.  Please do not try to confuse the issue by bringing in what Windows does, as I am not using Windows when I can avoid it.  Also, who said they had localhost for the outgoing interface?

johnpoz, you know more than I do about pfSense and I assume you also know a lot about networking.  However, here I'm seeing what I referred to before, where someone takes what's considered common knowledge and assumes it's gospel, without actually exploring the evidence and facts.  In this thread, I earlier demonstrated with dig that the Google DNS was being used, when pfSense failed.  I then posted a link to the resolv.conf man page, which describes how things work on Linux.  Yet, despite that, you still insisted I was wrong and told me how it works in Windows, even though that is entirely irrelevant to my situation.  Now, please go back and look at what I said, what the man page says and what the capture shows and show me if I'm wrong.  Based on what I've provided, I am not.

BTW, I demonstrated to myself, years ago, that the DNS list works exactly as I described.  That is the first DNS server is tried, then the 2nd, etc..  Please note, this is on Linux, not Windows.

Offline Flamethrower

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #41 on: November 09, 2017, 10:53:24 am »
Hi to all!

I want to share a little observation about issue with DNS Resolver. I updated my PFSENSE couple of weeks ago without any issues, but today DNS Resolver plays dead for me. After some tests I seen Resolver service is  stopped and tried to put it up, with no effect, it just not starting, no errors, nothing. So I tried to switch off DNSSEC for a test, and when I clicked "Save", I've got this:

------
The following input errors were detected:

    The generated config file cannot be parsed by unbound. Please correct the following errors:
    /var/unbound/test/unbound_server.pem: No such file or directory
    [1510244677] unbound-checkconf[89948:0] fatal error: auto-trust-anchor-file: "/var/unbound/test/root.key" does not exist in chrootdir /var/unbound
-----

And - what a surpise! - there is no such folder /var/unbound/test/ 
So, I think we nailed the root of a problem.

P.S. Sorry for my bad English

Offline JKnott

  • Hero Member
  • *****
  • Posts: 895
  • Karma: +29/-4
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #42 on: November 09, 2017, 10:55:45 am »
^^^^
I have also seen that error message and other issues.  2.4.1 took a working 2.4.0 installation and caused it to fail.  Why should that happen?

Offline kejianshi

  • Hero Member
  • *****
  • Posts: 4902
  • Karma: +194/-40
  • Debugging...
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #43 on: November 09, 2017, 11:07:19 am »
I can verify that directory is also non-existent for me.  Not sure why or if it is the cause of the breakage, but it is still broken.  (1 VM Only is like this)

However, the file also seems to not exist on any of my machines that work just fine. 
« Last Edit: November 09, 2017, 11:14:25 am by kejianshi »

Offline Flamethrower

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: 2.4.1: local DNS not working
« Reply #44 on: November 10, 2017, 03:26:11 am »
a little update:

after another attempt to start Reslover I've got in log this one:

----
Nov 10 12:14:31    unbound    31406:0    notice: init module 0: validator
Nov 10 12:14:31    unbound    31406:0    error: failed to read /root.key
Nov 10 12:14:31    unbound    31406:0    error: error reading auto-trust-anchor-file: /var/unbound/root.key
Nov 10 12:14:31    unbound    31406:0    error: validator: error in trustanchors config
Nov 10 12:14:31    unbound    31406:0    error: validator: could not apply configuration settings.
Nov 10 12:14:31    unbound    31406:0    error: module init for module validator failed
Nov 10 12:14:31    unbound    31406:0    fatal error: failed to setup modules
----

After that I checked  /var/unbound/root.key and found it zero sized. I tried to rebuild the /var/unbound/root.key but on each startup attempt unbound truncates it again to 0 than fails to start.