The pfSense Store

Author Topic: Periodic since 2.2 pages load blank, certs invalid  (Read 14857 times)

0 Members and 1 Guest are viewing this topic.

Online kejianshi

  • Hero Member
  • *****
  • Posts: 4928
  • Karma: +196/-40
  • Debugging...
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #90 on: February 09, 2015, 08:00:39 pm »
Have you guys considered setting the advanced settings that prevent excessive replies and purge things when excessive replies and poisoning is possibly happening?

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9098
  • Karma: +1037/-307
    • View Profile
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline agreenfield1

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #92 on: February 09, 2015, 08:11:01 pm »
Yup.

Unbound starts returning SERVFAIL for random domains after querying at least one or both of those hostnames.  I'm doing it one more time at unbound log level 5.



This sounds like the manifestation of the issue with DNSSEC enabled.  At some point, you may want to try it again with DNSSEC disabled; you should then see all domains being resolved to a hostile IP, bad certs for https, etc. like reported in the beginning of this thread.  Not sure if this would help with the diagnostics.

Offline Trel

  • Sr. Member
  • ****
  • Posts: 368
  • Karma: +11/-1
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #93 on: February 09, 2015, 08:25:33 pm »
Yup.

Unbound starts returning SERVFAIL for random domains after querying at least one or both of those hostnames.  I'm doing it one more time at unbound log level 5.



This sounds like the manifestation of the issue with DNSSEC enabled.  At some point, you may want to try it again with DNSSEC disabled; you should then see all domains being resolved to a hostile IP, bad certs for https, etc. like reported in the beginning of this thread.  Not sure if this would help with the diagnostics.

Yes, DNSSEC was the change I made that made it go from the bad domain resolutions to the failure to resolve at all.

I've had both issues and DNSSEC was the difference between which one I got.

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9098
  • Karma: +1037/-307
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #94 on: February 09, 2015, 08:58:51 pm »
Uncheck DNSSEC, Save, Apply....  There is most certainly something here....

Code: [Select]
$ dig ns3.csof.net

; <<>> DiG 9.8.3-P1 <<>> ns3.csof.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1411
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 5, ADDITIONAL: 3

;; QUESTION SECTION:
;ns3.csof.net. IN A

;; ANSWER SECTION:
ns3.csof.net. 600 IN A 195.22.26.199

;; AUTHORITY SECTION:
csof.net. 1792 IN NS ns62.domaincontrol.com.
csof.net. 1792 IN NS ns61.domaincontrol.com.
csof.net. 1792 IN NS ns4.csof.net.
csof.net. 1792 IN NS ns2.csof.net.
csof.net. 1792 IN NS ns1.csof.net.

;; ADDITIONAL SECTION:
ns4.csof.net. 1792 IN A 54.72.8.183
ns2.csof.net. 1793 IN A 212.6.183.201
ns1.csof.net. 1792 IN A 54.77.72.254

;; Query time: 29 msec
;; SERVER: 192.168.223.1#53(192.168.223.1)
;; WHEN: Mon Feb  9 18:48:25 2015
;; MSG SIZE  rcvd: 203

$ dig api-nyc01.exip.org

; <<>> DiG 9.8.3-P1 <<>> api-nyc01.exip.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14360
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;api-nyc01.exip.org. IN A

;; ANSWER SECTION:
api-nyc01.exip.org. 10 IN A 195.22.26.248

;; Query time: 206 msec
;; SERVER: 192.168.223.1#53(192.168.223.1)
;; WHEN: Mon Feb  9 18:48:35 2015
;; MSG SIZE  rcvd: 52

$ dig www.pfsense.org

; <<>> DiG 9.8.3-P1 <<>> www.pfsense.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51593
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 0

;; QUESTION SECTION:
;www.pfsense.org. IN A

;; ANSWER SECTION:
www.pfsense.org. 10 IN A 195.22.26.248

;; AUTHORITY SECTION:
org. 172779 IN NS ns1.csof.net.
org. 172779 IN NS ns2.csof.net.
org. 172779 IN NS ns3.csof.net.
org. 172779 IN NS ns4.csof.net.

;; Query time: 159 msec
;; SERVER: 192.168.223.1#53(192.168.223.1)
;; WHEN: Mon Feb  9 18:48:39 2015
;; MSG SIZE  rcvd: 129
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9098
  • Karma: +1037/-307
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #95 on: February 09, 2015, 09:04:07 pm »
looks like unbound is allowing itself to get polluted by this nonsense:

Code: [Select]
gridbug:etc cjl$ dig @ns1.csof.net. com. ns

; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. com. ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31834
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;com. IN NS

;; ANSWER SECTION:
com. 172800 IN NS ns1.csof.net.
com. 172800 IN NS ns2.csof.net.
com. 172800 IN NS ns3.csof.net.
com. 172800 IN NS ns4.csof.net.

;; ADDITIONAL SECTION:
ns1.csof.net. 100 IN A 54.77.72.254
ns2.csof.net. 100 IN A 212.6.183.201
ns3.csof.net. 100 IN A 195.22.26.199
ns4.csof.net. 100 IN A 54.72.8.183

;; Query time: 163 msec
;; SERVER: 54.77.72.254#53(54.77.72.254)
;; WHEN: Mon Feb  9 18:58:46 2015
;; MSG SIZE  rcvd: 165

gridbug:etc cjl$ dig @ns1.csof.net. . ns

; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. . ns
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
gridbug:etc cjl$ dig @ns1.csof.net. net. ns

; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. net. ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23620
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;net. IN NS

;; ANSWER SECTION:
net. 172800 IN NS ns1.csof.net.
net. 172800 IN NS ns2.csof.net.
net. 172800 IN NS ns3.csof.net.
net. 172800 IN NS ns4.csof.net.

;; ADDITIONAL SECTION:
ns1.csof.net. 100 IN A 54.77.72.254
ns2.csof.net. 100 IN A 212.6.183.201
ns3.csof.net. 100 IN A 195.22.26.199
ns4.csof.net. 100 IN A 54.72.8.183

;; Query time: 161 msec
;; SERVER: 54.77.72.254#53(54.77.72.254)
;; WHEN: Mon Feb  9 18:59:19 2015
;; MSG SIZE  rcvd: 162

gridbug:etc cjl$ dig @ns1.csof.net. com. ns

; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. com. ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57675
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;com. IN NS

;; ANSWER SECTION:
com. 172800 IN NS ns1.csof.net.
com. 172800 IN NS ns2.csof.net.
com. 172800 IN NS ns3.csof.net.
com. 172800 IN NS ns4.csof.net.

;; ADDITIONAL SECTION:
ns1.csof.net. 100 IN A 54.77.72.254
ns2.csof.net. 100 IN A 212.6.183.201
ns3.csof.net. 100 IN A 195.22.26.199
ns4.csof.net. 100 IN A 54.72.8.183

;; Query time: 164 msec
;; SERVER: 54.77.72.254#53(54.77.72.254)
;; WHEN: Mon Feb  9 18:59:23 2015
;; MSG SIZE  rcvd: 165

gridbug:etc cjl$ dig @ns1.csof.net. org. ns

; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. org. ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20295
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;org. IN NS

;; ANSWER SECTION:
org. 172800 IN NS ns1.csof.net.
org. 172800 IN NS ns2.csof.net.
org. 172800 IN NS ns3.csof.net.
org. 172800 IN NS ns4.csof.net.

;; ADDITIONAL SECTION:
ns1.csof.net. 100 IN A 54.77.72.254
ns2.csof.net. 100 IN A 212.6.183.201
ns3.csof.net. 100 IN A 195.22.26.199
ns4.csof.net. 100 IN A 54.72.8.183

;; Query time: 162 msec
;; SERVER: 54.77.72.254#53(54.77.72.254)
;; WHEN: Mon Feb  9 18:59:27 2015
;; MSG SIZE  rcvd: 165

gridbug:etc cjl$ dig @ns1.csof.net. edu. ns

; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. edu. ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28770
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;edu. IN NS

;; ANSWER SECTION:
edu. 172800 IN NS ns1.csof.net.
edu. 172800 IN NS ns2.csof.net.
edu. 172800 IN NS ns3.csof.net.
edu. 172800 IN NS ns4.csof.net.

;; ADDITIONAL SECTION:
ns1.csof.net. 100 IN A 54.77.72.254
ns2.csof.net. 100 IN A 212.6.183.201
ns3.csof.net. 100 IN A 195.22.26.199
ns4.csof.net. 100 IN A 54.72.8.183

;; Query time: 161 msec
;; SERVER: 54.77.72.254#53(54.77.72.254)
;; WHEN: Mon Feb  9 18:59:31 2015
;; MSG SIZE  rcvd: 165

gridbug:etc cjl$ dig @ns1.csof.net. gov. ns

; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. gov. ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46880
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;gov. IN NS

;; ANSWER SECTION:
gov. 172800 IN NS ns1.csof.net.
gov. 172800 IN NS ns2.csof.net.
gov. 172800 IN NS ns3.csof.net.
gov. 172800 IN NS ns4.csof.net.

;; ADDITIONAL SECTION:
ns1.csof.net. 100 IN A 54.77.72.254
ns2.csof.net. 100 IN A 212.6.183.201
ns3.csof.net. 100 IN A 195.22.26.199
ns4.csof.net. 100 IN A 54.72.8.183

;; Query time: 162 msec
;; SERVER: 54.77.72.254#53(54.77.72.254)
;; WHEN: Mon Feb  9 18:59:34 2015
;; MSG SIZE  rcvd: 165

gridbug:etc cjl$ dig @ns1.csof.net. mil. ns

; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. mil. ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21791
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;mil. IN NS

;; ANSWER SECTION:
mil. 172800 IN NS ns1.csof.net.
mil. 172800 IN NS ns2.csof.net.
mil. 172800 IN NS ns3.csof.net.
mil. 172800 IN NS ns4.csof.net.

;; ADDITIONAL SECTION:
ns1.csof.net. 100 IN A 54.77.72.254
ns2.csof.net. 100 IN A 212.6.183.201
ns3.csof.net. 100 IN A 195.22.26.199
ns4.csof.net. 100 IN A 54.72.8.183

;; Query time: 160 msec
;; SERVER: 54.77.72.254#53(54.77.72.254)
;; WHEN: Mon Feb  9 18:59:40 2015
;; MSG SIZE  rcvd: 165

gridbug:etc cjl$ dig @ns1.csof.net. us. ns

; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. us. ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5930
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;us. IN NS

;; ANSWER SECTION:
us. 172800 IN NS ns1.csof.net.
us. 172800 IN NS ns2.csof.net.
us. 172800 IN NS ns3.csof.net.
us. 172800 IN NS ns4.csof.net.

;; ADDITIONAL SECTION:
ns1.csof.net. 100 IN A 54.77.72.254
ns2.csof.net. 100 IN A 212.6.183.201
ns3.csof.net. 100 IN A 195.22.26.199
ns4.csof.net. 100 IN A 54.72.8.183

;; Query time: 161 msec
;; SERVER: 54.77.72.254#53(54.77.72.254)
;; WHEN: Mon Feb  9 18:59:43 2015
;; MSG SIZE  rcvd: 164
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9098
  • Karma: +1037/-307
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #96 on: February 09, 2015, 09:14:27 pm »
Back to dnsmasq for me until someone says something.
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline Trel

  • Sr. Member
  • ****
  • Posts: 368
  • Karma: +11/-1
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #97 on: February 09, 2015, 09:35:19 pm »
I don't know which I should be.

Upset that unbound is doing this
Relieved that everything I was seeing can be reproduced

Offline Derelict

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9098
  • Karma: +1037/-307
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #98 on: February 09, 2015, 09:50:22 pm »
You should be thankful you found something that made it easily-reproducible.  No need to be mad.  Things happen - Look at BIND's history.  And you can always just run dnsmasq in the meantime even though that's the exact opposite advice you got at first.  :/
Las Vegas, Nevada, USA
Use this diagram to describe your issue.
The pfSense Book is now available for just $24.70!
Do Not PM For Help! NO_WAN_EGRESSTM

Offline agreenfield1

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #99 on: February 09, 2015, 10:23:49 pm »
I'm really out of my league trying to diagnose this, but here is the output of 'unbound-control -c /var/unbound/unbound.conf lookup slashdot.org', when the issue is occuring with DNSSEC enabled:

Code: [Select]
unbound-control -c /var/unbound/unbound.conf lookup slashdot.org
The following name servers are used for lookup of slashdot.org.
;rrset 85785 4 0 7 0
org. 172185 IN NS ns1.csof.net.
org. 172185 IN NS ns2.csof.net.
org. 172185 IN NS ns3.csof.net.
org. 172185 IN NS ns4.csof.net.
;rrset 83879 2 1 11 4
org. 83879 IN DS 21366 7 1 E6C1716CFB6BDC84E84CE1AB5510DAC69173B5B2
org. 83879 IN DS 21366 7 2 96EEB2FFD9B00CD4694E78278B5EFDAB0A80446567B69F634DA078F0D90F01BA
org. 83879 IN RRSIG DS 8 1 86400 20150219170000 20150209160000 16665 . RxYpI0BzYpGTE/PjRQdR4SZaxlvXCja3SJyx10JagTfz20gnltl4ar94GOwp8bA/ktY/7JxMoJvzCTAtcsGaTGRv04yDHr7WaydMxZuPCP9YT9Ixc+fX9IAZlSfwLCkBQgiC0mVeRiq+LmbIJhI2grJbTtvy96O9mipAqkFR42g= ;{id = 16665}
;rrset 1185 1 0 3 0
ns4.csof.net. 1185 IN A 54.72.8.183
;rrset 197 1 0 8 0
ns3.csof.net. 197 IN A 195.22.26.199
;rrset 1185 1 0 8 0
ns2.csof.net. 1185 IN A 212.6.183.201
;rrset 1185 1 0 8 0
ns1.csof.net. 1185 IN A 54.77.72.254
Delegation with 4 names, of which 4 can be examined to query further addresses.
It provides 4 IP addresses.
54.77.72.254    NoDNSSEC rto 344 msec, ttl 286, ping 84 var 65 rtt 344, tA 0, tAAAA 0, tother 0, EDNS 0 probed.
212.6.183.201    NoDNSSEC rto 365 msec, ttl 286, ping 69 var 74 rtt 365, tA 0, tAAAA 0, tother 0, EDNS 0 probed.
195.22.26.199    rto 96256 msec, ttl 286, ping 0 var 94 rtt 376, tA 3, tAAAA 3, tother 3, EDNS 0 assumed.
54.72.8.183      NoDNSSEC rto 315 msec, ttl 286, ping 99 var 54 rtt 315, tA 0, tAAAA 0, tother 0, EDNS 0 probed.

And then after restarting unbound:

Code: [Select]
unbound-control -c /var/unbound/unbound.conf lookup slashdot.org
The following name servers are used for lookup of slashdot.org.
;rrset 86374 6 0 2 0
org. 86374 IN NS a0.org.afilias-nst.info.
org. 86374 IN NS a2.org.afilias-nst.info.
org. 86374 IN NS b0.org.afilias-nst.org.
org. 86374 IN NS b2.org.afilias-nst.org.
org. 86374 IN NS c0.org.afilias-nst.info.
org. 86374 IN NS d0.org.afilias-nst.org.
;rrset 86374 2 1 2 0
org. 86374 IN DS 21366 7 1 E6C1716CFB6BDC84E84CE1AB5510DAC69173B5B2
org. 86374 IN DS 21366 7 2 96EEB2FFD9B00CD4694E78278B5EFDAB0A80446567B69F634DA078F0D90F01BA
org. 86374 IN RRSIG DS 8 1 86400 20150219170000 20150209160000 16665 . RxYpI0BzYpGTE/PjRQdR4SZaxlvXCja3SJyx10JagTfz20gnltl4ar94GOwp8bA/ktY/7JxMoJvzCTAtcsGaTGRv04yDHr7WaydMxZuPCP9YT9Ixc+fX9IAZlSfwLCkBQgiC0mVeRiq+LmbIJhI2grJbTtvy96O9mipAqkFR42g= ;{id = 16665}
;rrset 86374 1 0 1 0
d0.org.afilias-nst.org. 172774 IN A 199.19.57.1
;rrset 86374 1 0 1 0
d0.org.afilias-nst.org. 172774 IN AAAA 2001:500:f::1
;rrset 86374 1 0 1 0
c0.org.afilias-nst.info. 172774 IN A 199.19.53.1
;rrset 86374 1 0 1 0
c0.org.afilias-nst.info. 172774 IN AAAA 2001:500:b::1
;rrset 86374 1 0 1 0
b2.org.afilias-nst.org. 172774 IN A 199.249.120.1
;rrset 86374 1 0 1 0
b2.org.afilias-nst.org. 172774 IN AAAA 2001:500:48::1
;rrset 86374 1 0 1 0
b0.org.afilias-nst.org. 172774 IN A 199.19.54.1
;rrset 86374 1 0 1 0
b0.org.afilias-nst.org. 172774 IN AAAA 2001:500:c::1
;rrset 86374 1 0 1 0
a2.org.afilias-nst.info. 172774 IN A 199.249.112.1
;rrset 86374 1 0 1 0
a2.org.afilias-nst.info. 172774 IN AAAA 2001:500:40::1
;rrset 86374 1 0 1 0
a0.org.afilias-nst.info. 172774 IN A 199.19.56.1
;rrset 86374 1 0 1 0
a0.org.afilias-nst.info. 172774 IN AAAA 2001:500:e::1
Delegation with 6 names, of which 0 can be examined to query further addresses.
It provides 12 IP addresses.
2001:500:e::1    not in infra cache.
199.19.56.1      not in infra cache.
2001:500:40::1  not in infra cache.
199.249.112.1    not in infra cache.
2001:500:c::1    not in infra cache.
199.19.54.1      not in infra cache.
2001:500:48::1  not in infra cache.
199.249.120.1    rto 356 msec, ttl 874, ping 8 var 87 rtt 356, tA 0, tAAAA 0, tother 0, EDNS 0 probed.
2001:500:b::1    not in infra cache.
199.19.53.1      rto 482 msec, ttl 874, ping 22 var 115 rtt 482, tA 0, tAAAA 0, tother 0, EDNS 0 probed.
2001:500:f::1    not in infra cache.
199.19.57.1      not in infra cache.

Online kejianshi

  • Hero Member
  • *****
  • Posts: 4928
  • Karma: +196/-40
  • Debugging...
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #100 on: February 09, 2015, 10:48:30 pm »
I'll ask again...

In Services: DNS Resolver: Advanced

Have you tried checking:

Harden Glue

Harden DNSSEC data

Unwanted Reply Threshold (10 million)

etc...   ???

Offline cmb

  • Hero Member
  • *****
  • Posts: 11230
  • Karma: +893/-7
    • View Profile
    • Chris Buechler
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #101 on: February 09, 2015, 11:00:48 pm »
Have you tried checking:

Harden Glue

Harden DNSSEC data

These two in particular, if you don't have them enabled, enable them. I changed things to enable both those by default, and we'll add config upgrade code to turn those on for anyone who doesn't have them enabled upon upgrade to 2.2.1.

Online kejianshi

  • Hero Member
  • *****
  • Posts: 4928
  • Karma: +196/-40
  • Debugging...
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #102 on: February 09, 2015, 11:06:13 pm »
Yeah - I went on a voyage of discovery to find a combo that worked...
Less a matter of being smart.  More a matter of random experimentation.

Even reading and searching online yielded little results.

But yeah - Fixes alot.

I'm glad you guys are making it defaults.

Offline doktornotor

  • Hero Member
  • *****
  • Posts: 8553
  • Karma: +958/-278
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #103 on: February 10, 2015, 02:44:44 am »
Couple more of the hardened stuff here: https://forum.pfsense.org/index.php?topic=88466.msg488511#msg488511

- harden-referral-path is hardcoded to no in unbound.inc ATM  :(
- harden-below-nxdomain can be set via the advanced config

(created a feature request for the two above)

- there's also this use-caps-for-id thing (patch to expose in GUI in 4205 or stick use-caps-for-id: yes to advanced config.

Another thing that makes me wonder... dnsmasq AFAICT is not compiled with DNSSEC support at all on pfSense even though it's apparently supported now. Hmmm. And of course, no matter what you are completely screwed with unsigned domains, and the DNS protocol is a piece of insecure crap.

A couple of suggestions for your own servers:
- sign your domains (ahem... why's pfsense.org not signed?!)
- if your registrar does not allow you to do so, switch registrar
- if the TLD you are using is not signed, choose a different one
- use DANE/TLSA for your critical servers at least.

Some useful tools:
« Last Edit: February 10, 2015, 04:37:49 am by doktornotor »
Do NOT PM for help!

Offline swix

  • Jr. Member
  • **
  • Posts: 26
  • Karma: +1/-0
    • View Profile
Re: Periodic since 2.2 pages load blank, certs invalid
« Reply #104 on: February 10, 2015, 04:04:57 am »
Update for our situation (only dns active, harden glue settings not (yet) updated).
The situation happened again a few minutes ago, and here it is how it looked in the resolver.log (first time "csof" is visible today)  :

Code: [Select]
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:18 pf unbound: [24807:1] info: response for 239.75.246.46.in-addr.arpa. PTR IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <75.246.46.in-addr.arpa.> 91.213.246.6#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns1.transitionalprotocol.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns2.transitionalprotocol.net. A IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving f.gtld-servers.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving g.gtld-servers.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving k.gtld-servers.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving l.gtld-servers.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns2.transitionalprotocol.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving h.gtld-servers.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving i.gtld-servers.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns1.transitionalprotocol.net. A IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving m.gtld-servers.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving d.gtld-servers.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: response for a.ns.portlane.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <portlane.net.> 80.67.0.6#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:18 pf unbound: [24807:1] info: response for b.ns.portlane.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <portlane.net.> 80.67.0.6#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:18 pf unbound: [24807:1] info: response for 125.10.113.188.in-addr.arpa. PTR IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <10.113.188.in-addr.arpa.> 80.240.240.2#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving 113.188.in-addr.arpa. DS IN
Feb 10 10:03:18 pf unbound: [24807:1] notice: sendto failed: Operation not permitted
Feb 10 10:03:18 pf unbound: [24807:1] notice: remote address is 2001:500:13::c7d4:35 port 53
Feb 10 10:03:18 pf unbound: [24807:1] info: error sending query to auth server 2001:500:13::c7d4:35 port 53
Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns1.transitionalprotocol.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.> 192.52.178.30#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns2.csof.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns1.csof.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving e.gtld-servers.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving d.gtld-servers.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.transitionalprotocol.net. A IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.> 192.52.178.30#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL
Feb 10 10:03:18 pf unbound: [24807:0] info: response for 125.74.1.116.in-addr.arpa. PTR IN
Feb 10 10:03:18 pf unbound: [24807:0] info: reply from <1.116.in-addr.arpa.> 202.103.224.69#53
Feb 10 10:03:18 pf unbound: [24807:0] info: query response was NXDOMAIN ANSWER
Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.transitionalprotocol.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.> 192.26.92.30#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL
Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.transitionalprotocol.net. A IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <transitionalprotocol.net.> 212.6.183.201#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns1.transitionalprotocol.net. A IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.> 192.31.80.30#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL
Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.csof.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.> 192.43.172.30#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL
Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.transitionalprotocol.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <transitionalprotocol.net.> 212.6.183.201#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was nodata ANSWER
Feb 10 10:03:18 pf unbound: [24807:1] info: response for 113.188.in-addr.arpa. DS IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <188.in-addr.arpa.> 199.212.0.53#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was nodata ANSWER
Feb 10 10:03:18 pf unbound: [24807:1] info: NSEC RRset for the referral proved not a delegation point
Feb 10 10:03:18 pf unbound: [24807:1] info: NSEC RRset for the referral proved no DS.
Feb 10 10:03:18 pf unbound: [24807:1] info: Verified that unsigned response is INSECURE
Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns1.transitionalprotocol.net. A IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <transitionalprotocol.net.> 54.77.72.254#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns1.csof.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.> 192.26.92.30#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL
Feb 10 10:03:18 pf unbound: [24807:1] info: response for 239.75.246.46.in-addr.arpa. PTR IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <239.75.246.46.in-addr.arpa.> 195.22.26.248#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:18 pf unbound: [24807:1] info: resolving 246.46.in-addr.arpa. DS IN
Feb 10 10:03:18 pf unbound: [24807:1] info: NSEC RRset for the referral proved not a delegation point
Feb 10 10:03:18 pf unbound: [24807:1] info: NSEC RRset for the referral proved no DS.
Feb 10 10:03:18 pf unbound: [24807:1] info: Verified that unsigned response is INSECURE
Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.csof.net. AAAA IN
Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <csof.net.> 208.109.255.32#53
Feb 10 10:03:18 pf unbound: [24807:1] info: query response was nodata ANSWER
Feb 10 10:03:18 pf unbound: [24807:0] info: response for 166.31.215.188.in-addr.arpa. PTR IN
Feb 10 10:03:18 pf unbound: [24807:0] info: reply from <31.215.188.in-addr.arpa.> 89.39.166.2#53
Feb 10 10:03:18 pf unbound: [24807:0] info: query response was THROWAWAY
Feb 10 10:03:19 pf unbound: [24807:1] info: response for ns1.csof.net. AAAA IN
Feb 10 10:03:19 pf unbound: [24807:1] info: reply from <csof.net.> 208.109.255.32#53
Feb 10 10:03:19 pf unbound: [24807:1] info: query response was nodata ANSWER
Feb 10 10:03:19 pf unbound: [24807:1] info: resolving 205.112.194.173.in-addr.arpa. PTR IN
Feb 10 10:03:19 pf unbound: [24807:1] info: resolving ns2.google.com. AAAA IN
Feb 10 10:03:19 pf unbound: [24807:1] info: resolving ns4.google.com. AAAA IN
Feb 10 10:03:19 pf unbound: [24807:1] info: resolving ns1.google.com. AAAA IN
Feb 10 10:03:19 pf unbound: [24807:1] info: response for ns1.transitionalprotocol.net. AAAA IN
Feb 10 10:03:19 pf unbound: [24807:1] info: reply from <transitionalprotocol.net.> 212.6.183.201#53
Feb 10 10:03:19 pf unbound: [24807:1] info: query response was nodata ANSWER
Feb 10 10:03:19 pf unbound: [24807:1] info: response for 205.112.194.173.in-addr.arpa. PTR IN
Feb 10 10:03:19 pf unbound: [24807:1] info: reply from <194.173.in-addr.arpa.> 216.239.36.10#53
Feb 10 10:03:19 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:20 pf unbound: [24807:1] info: resolving www.insign.ch. A IN
Feb 10 10:03:20 pf unbound: [24807:1] info: resolving dns2.insign.ch. AAAA IN
Feb 10 10:03:20 pf unbound: [24807:1] info: resolving dns1.insign.ch. AAAA IN
Feb 10 10:03:20 pf unbound: [24807:1] info: response for www.insign.ch. A IN
Feb 10 10:03:20 pf unbound: [24807:1] info: reply from <insign.ch.> 46.175.9.120#53
Feb 10 10:03:20 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:20 pf unbound: [24807:1] info: NSEC3s for the referral proved no DS.
Feb 10 10:03:20 pf unbound: [24807:1] info: Verified that unsigned response is INSECURE
Feb 10 10:03:20 pf unbound: [24807:1] info: resolving 64.94.172.95.in-addr.arpa. PTR IN
Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns-b.pnap.net. AAAA IN
Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns-c.pnap.net. AAAA IN
Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns-a.pnap.net. AAAA IN
Feb 10 10:03:20 pf unbound: [24807:1] info: resolving 185.112.194.173.in-addr.arpa. PTR IN
Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns3.google.com. AAAA IN
Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns4.google.com. AAAA IN
Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns1.google.com. AAAA IN
Feb 10 10:03:20 pf unbound: [24807:1] info: response for 185.112.194.173.in-addr.arpa. PTR IN
Feb 10 10:03:20 pf unbound: [24807:1] info: reply from <194.173.in-addr.arpa.> 216.239.32.10#53
Feb 10 10:03:20 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:20 pf unbound: [24807:1] info: response for 64.94.172.95.in-addr.arpa. PTR IN
Feb 10 10:03:20 pf unbound: [24807:1] info: reply from <94.172.95.in-addr.arpa.> 64.95.61.4#53
Feb 10 10:03:41 pf unbound: [24807:0] info: query response was REFERRAL
Feb 10 10:03:41 pf unbound: [24807:0] info: resolving dns4.bigrock.in. AAAA IN
Feb 10 10:03:41 pf unbound: [24807:0] info: resolving dns2.bigrock.in. AAAA IN
Feb 10 10:03:41 pf unbound: [24807:0] info: resolving asia1.akam.net. AAAA IN
Feb 10 10:03:41 pf unbound: [24807:0] notice: sendto failed: Operation not permitted
Feb 10 10:03:41 pf unbound: [24807:0] notice: remote address is 2600:1401:2::43 port 53
Feb 10 10:03:41 pf unbound: [24807:0] info: error sending query to auth server 2600:1401:2::43 port 53
Feb 10 10:03:41 pf unbound: [24807:1] info: response for ns0.mirasystem.net. A IN
Feb 10 10:03:41 pf unbound: [24807:1] info: reply from <net.> 54.72.8.183#53
Feb 10 10:03:41 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:42 pf unbound: [24807:0] info: response for asia1.akam.net. AAAA IN
Feb 10 10:03:42 pf unbound: [24807:0] info: reply from <akam.net.> 184.85.248.67#53
Feb 10 10:03:42 pf unbound: [24807:0] info: query response was nodata ANSWER
Feb 10 10:03:42 pf unbound: [24807:0] info: response for dns2.bigrock.in. AAAA IN
Feb 10 10:03:42 pf unbound: [24807:0] info: reply from <bigrock.in.> 2.22.230.64#53
Feb 10 10:03:42 pf unbound: [24807:0] info: query response was nodata ANSWER
Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns1.epn.ru. A IN
Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <EPN.RU.> 195.22.26.248#53
Feb 10 10:03:42 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns0.epn.ru. AAAA IN
Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <epn.ru.> 195.22.26.248#53
Feb 10 10:03:42 pf unbound: [24807:1] info: query response was nodata ANSWER
Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns0.epn.ru. A IN
Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <EPN.RU.> 195.22.26.248#53
Feb 10 10:03:42 pf unbound: [24807:1] info: query response was ANSWER
Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns1.epn.ru. AAAA IN
Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <EPN.RU.> 195.22.26.248#53
Feb 10 10:03:42 pf unbound: [24807:1] info: query response was nodata ANSWER
Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns1.hn.cnc.cn. AAAA IN
Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <cnc.cn.> 210.52.207.2#53
Feb 10 10:03:42 pf unbound: [24807:1] info: query response was REFERRAL
Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns1.hn.cnc.cn. A IN
Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <cnc.cn.> 210.52.207.2#53
Feb 10 10:03:42 pf unbound: [24807:1] info: query response was REFERRAL
Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns2.hn.cnc.cn. A IN
Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <cnc.cn.> 210.52.207.2#53
Feb 10 10:03:42 pf unbound: [24807:1] info: query response was REFERRAL
Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns2.hn.cnc.cn. AAAA IN

At the end, all requests get resolved as "195.22.26.248".
Trigger request couldn't yet be found (the log rule on port 53 for requests not going over the router returned nothing at this moment).

So when it starts, every single request then go over one of these ns1.csof.net name server, and get answered as if csof.net was a root server, but with wrong data.  For example:

Normal case when querying the pfsense resolver (when everything is ok) :

Code: [Select]
om@ompc:~> dig -t ns @192.168.1.100 ch.

; <<>> DiG 9.9.5-rpz2+rl.14038.05-P1 <<>> -t ns @192.168.1.100 ch.
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62805
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 14

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ch.                            IN      NS

;; ANSWER SECTION:
ch.                     85292   IN      NS      c.nic.ch.
ch.                     85292   IN      NS      b.nic.ch.
ch.                     85292   IN      NS      e.nic.ch.
ch.                     85292   IN      NS      d.nic.ch.
ch.                     85292   IN      NS      a.nic.ch.
ch.                     85292   IN      NS      h.nic.ch.
ch.                     85292   IN      NS      f.nic.ch.

;; ADDITIONAL SECTION:
a.nic.ch.               171692  IN      A       130.59.1.80
a.nic.ch.               171692  IN      AAAA    2001:620::4
b.nic.ch.               171692  IN      A       130.59.211.10
b.nic.ch.               171692  IN      AAAA    2001:620::5
c.nic.ch.               171692  IN      A       147.28.0.39
c.nic.ch.               171692  IN      AAAA    2001:418:1::39
d.nic.ch.               171692  IN      A       200.160.0.5
d.nic.ch.               171692  IN      AAAA    2001:12ff:0:a20::5
e.nic.ch.               171692  IN      A       194.0.17.1
e.nic.ch.               171692  IN      AAAA    2001:678:3::1
f.nic.ch.               171692  IN      A       194.146.106.10
f.nic.ch.               171692  IN      AAAA    2001:67c:1010:2::53
h.nic.ch.               171692  IN      A       194.42.48.120

;; Query time: 0 msec
;; SERVER: 192.168.1.100#53(192.168.1.100)
;; WHEN: Tue Feb 10 10:55:52 CET 2015
;; MSG SIZE  rcvd: 427


When querying this ns1.csof.net server :

Code: [Select]
om@ompc:~> dig -t ns @ns1.csof.net ch.

; <<>> DiG 9.9.5-rpz2+rl.14038.05-P1 <<>> -t ns @ns1.csof.net ch.
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45232
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;ch.                            IN      NS

;; ANSWER SECTION:
ch.                     172800  IN      NS      ns1.csof.net.
ch.                     172800  IN      NS      ns2.csof.net.
ch.                     172800  IN      NS      ns3.csof.net.
ch.                     172800  IN      NS      ns4.csof.net.

;; ADDITIONAL SECTION:
ns1.csof.net.           100     IN      A       54.77.72.254
ns2.csof.net.           100     IN      A       212.6.183.201
ns3.csof.net.           100     IN      A       195.22.26.199
ns4.csof.net.           100     IN      A       54.72.8.183

;; Query time: 54 msec
;; SERVER: 54.77.72.254#53(54.77.72.254)
;; WHEN: Tue Feb 10 10:45:26 CET 2015
;; MSG SIZE  rcvd: 164


I now increased the verbosity of unbound with "unbound-control -c /var/unbound/unbound.conf verbosity 4" and waiting for another hit...   If you have other suggestions in the mean time, please feel free.