Since update to FTLDNS Bogus DNSSEC results have increased

gecko · April 26, 2018, 12:51pm

Please follow the below template, it will help us to help you!

Expected Behaviour:

PiHole should be resolving DNS addresses correctly when using DNSSEC

Actual Behaviour:

Since the update to the FTLDNS Branch, I have experienced an increase in the number of Bogus results for DNSSEC resolution. This means that the page fails to load correctly. I feel like this is due to a misconfiguration in pihole FTLDNS, but can't be sure.

I have PiHole setup to use the Stubby daemon running on a local interface to resolve DNS-over-TLS from the Cloudflare 1.1.1.1 servers. Stubby is set up with DNSSEC. Supposedly Stubby doesn't need a trust anchor (the option for "configuration free DNSSEC" is selected in Stubby config).

Perhaps this is the issue, but I am sceptical of that, as it was working fine with very few Bogus results. Now, after an update to FTLDNS, Bogus results have increased.

Debug Token:

[Replace this text with the debug token provided from running pihole -d (or running the debug script through the web interface]

DanSchaper · April 26, 2018, 12:58pm

Did you remove dnsmasq when you installed FTLDNS? If not and it's still installed, can you tell us the version that was installed? (If you're on a Debian based distro then apt-cache policy dnsmasq should give you that information.) Thanks.

gecko · April 26, 2018, 1:15pm

No, I didn't uninstall (at least, I didn't actively uninstall it, to my knowledge!)

apt-cache policy dnsmasq has the following output:
dnsmasq: Installed: 2.76-5+rpt1+deb9u1 Candidate: 2.76-5+rpt1+deb9u1 Version table: *** 2.76-5+rpt1+deb9u1 500 500 http://archive.raspberrypi.org/debian stretch/main armhf Packages 100 /var/lib/dpkg/status 2.76-5+deb9u1 500 500 https://ftp.acc.umu.se/mirror/raspbian/raspbian stretch/main armhf Packages

gecko · April 26, 2018, 1:26pm

Also, forgot to add something that is perhaps relevant. The issue is very intermittent. One minute the resolution will fail, and in the PiHole web-interface query log it will say BOGUS under DNSSEC status. The next minute, after refreshing the page/trying again, the resolution will succeed, and the DNSSEC status will say INSECURE (the standard response for a domain that is not setup to use DNSSEC)

DL6ER · April 26, 2018, 2:32pm

One major difference with FTLDNS is that you're now running dnsmasq 2.79 under the hood. From the official CHANGELOG, I see quite a few changes concerning DNSSEC (only quoting DNSSEC relevant changes):

version 2.79
	Add support for Ed25519 signatures in DNSSEC validation.

	No longer support RSA/MD5 signatures in DNSSEC validation,
	since these are not secure. This behaviour is mandated in
	RFC-6944.

	Use SIGINT (instead of overloading SIGHUP) to turn on DNSSEC
	time validation when --dnssec-no-timecheck is in use.
	Note that this is an incompatible change from earlier releases.

	Fix for DNSSEC with wildcard-derived NSEC records.
	It's OK for NSEC records to be expanded from wildcards,
	but in that case, the proof of non-existence is only valid
	starting at the wildcard name, *.<domain> NOT the name expanded
	from the wildcard. Without this check it's possible for an
	attacker to craft an NSEC which wrongly proves non-existence.
	Thanks to Ralph Dolmans for finding this, and co-ordinating 
	the vulnerability tracking and fix release.
	CVE-2017-15107 applies.

version 2.77
	Fix problem with --dnssec-timestamp whereby receipt
	of SIGHUP would erroneously engage timestamp checking.
	Thanks to Kevin Darbyshire-Bryant for this work.

	Fix foobar in rrfilter code, that could cause malformed 
	replies, especially when DNSSEC validation on, and 
	the upstream server returns answer with the RRs in a 
	particular order. The only DNS server known to tickle
	this is Nominum's. Thanks to Dave Täht for spotting the
	bug and assisting in the fix.

I don't think anything of this should really be affecting you, but you never know...

Do you have any logs with stubby? Can you see in these logs if it told our resolver that the DNSSEC status was INSECURE but FTLDNS said BOGUS instead? Can you quote the relevant log lines from /var/log/pihole.log?

Without wanting to reject possible blame, I doubt that this can be the case. Our resolver uses the exact same configuration, your dnsmasq used before (we do not imply any configuration except from what is set up within /etc/dnsmasq.d/).

As I said above it would be very helpful to get some more logs.

Also - just for testing! - would you be willing to directly use an upstream DNS server for testing? By this, we could exclude that the problem is coming from stubby or its configuration.

jpgpi250 · April 26, 2018, 7:01pm

I've been mailing with Simon Kelley (developer of dnsmasq) for a while now, regarding this DNSSEC problem.
I'm NOT using FTLDNS, but regularly upgade dnsmasq, using the test releases (currently running dnsmasq2.80test2)
It appears this problem appeared in dnsmasq2.77 and is still there in dnsmasq2.79.

dnsmasq2.80test1 & dnsmasq2.80test2 attempt to fix this problem, however, they make the problem even more visible, so for now, I'm running dnsmasq with the setting ‘dnssec-check-unsigned=no’. I'm not sure this setting even works in earlier versions...

from the developer, regarding this setting:
'quote'
However, this is dangerous, since it allows dnsmasq to accept forged answers even to queries which should be signed.
'/quote'

'edit'
I'm running dnsmasq2.80test2 AND dnscrypt-proxy V2 and regulary get:
dnsmasq[2175]: nameserver 127.10.10.1 refused to do a recursive query

127.10.10.1 is the dnscrypt-proxy service, running on port 5551'

Anybody getting this error? Please report this!
'/edit'

To be continued.

gecko · April 27, 2018, 7:10am

Yeah, I worked out it probably has nothing to do with stubby. I found out yesterday from the stubby github that I shouldn’t even have DNSSEC enabled in the stubby config if I am only using stubby to forward onto dnsmasq, as it just increases overhead needlessly (I assumed DNSSEC had to be enabled in both pihole/dnsmasq and stubby for it to work).

I disabled DNSSEC in stubby, and DNSSEC still works correctly in piholle (raspberrypi.org comes back as secure, for example). But the problem with BOGUS results is still there. So many sites are coming back as BOGUS and faiul to resolve that I think I may have to disable DNSSEC for now, as it’s really interrupting my workflow having to keep waiting a few mins then refreshing pages for them to resolve.

The reason I felt it was an issue in FTLDNS is that I’ve had this configuration for a few weeks (since cloudflare 1.1.1.1 was made available on the first), but it’s only in the last week that I’ve had these issues, and since pihole FTLDNS branch is the only thing that’s been updated in that time, I assumed it was that.

gecko · April 27, 2018, 7:26am

I checked the stubby logs, and there was nothing in there I could see regarding DNSSEC queries. Like I said before, I was told by people on the stubby github that I should disable DNSSEC on stubby anyway, if I am forwarding requests on to dnsmasq/FTLDNS.

An example in the pihole log is from this morning. I was having an issue downloading an update on the Apple AppStore. This was in my log

Apr 27 09:00:52 dnsmasq[797]: 10060 192.168.150.90/51283 query[A] itunes.apple.com from 192.168.150.90.
Apr 27 09:00:52 dnsmasq[797]: 10060 192.168.150.90/51283 forwarded itunes.apple.com to 127.0.2.2.   
Apr 27 09:00:52 dnsmasq[797]: 10061 192.168.150.90/53464 query[A] p16-buy.itunes-apple.com.akadns.net from 192.168.150.90.    
Apr 27 09:00:52 dnsmasq[797]: 10061 192.168.150.90/53464 forwarded p16-buy.itunes-apple.com.akadns.net to 127.0.2.2.    
Apr 27 09:00:52 dnsmasq[797]: 10060 192.168.150.90/51283 validation itunes.apple.com is BOGUS`

127.0.2.2 is the stubby daemon, which resolves to 1.1.1.1 via DNS-over-TLS.

I can try bypassing stubby and going direct to 1.1.1.1 with DNSSEC enabled to see if that makes any difference, but from the sound of what jpgpi250 Is saying, this is a known issue with dnsmasq

gecko · April 27, 2018, 7:43am

He

Here are some more from this morning

Apr 27 09:31:40 dnsmasq[797]: 10431 192.168.150.89/57361 query[A] mvm.snapchat.com from 192.168.150.89
Apr 27 09:31:40 dnsmasq[797]: 10431 192.168.150.89/57361 forwarded mvm.snapchat.com to 127.0.2.2
Apr 27 09:31:40 dnsmasq[797]: 10432 192.168.150.89/63171 query[A] chat-gateway-prod.chat.snapchat.com from 192.168.150.89
Apr 27 09:31:40 dnsmasq[797]: 10432 192.168.150.89/63171 forwarded chat-gateway-prod.chat.snapchat.com to 127.0.2.2
Apr 27 09:31:40 dnsmasq[797]: 10433 192.168.150.89/60935 query[A] app-analytics.snapchat.com from 192.168.150.89
Apr 27 09:31:40 dnsmasq[797]: 10433 192.168.150.89/60935 forwarded app-analytics.snapchat.com to 127.0.2.2
Apr 27 09:31:40 dnsmasq[797]: 10432 192.168.150.89/63171 validation result is INSECURE
Apr 27 09:31:40 dnsmasq[797]: 10432 192.168.150.89/63171 reply chat-gateway-prod.chat.snapchat.com is 35.201.121.164
Apr 27 09:31:41 dnsmasq[797]: 10431 192.168.150.89/57361 validation mvm.snapchat.com is BOGUS
Apr 27 09:31:41 dnsmasq[797]: 10433 192.168.150.89/60935 validation app-analytics.snapchat.com is BOGUS
Apr 27 09:31:41 dnsmasq[797]: 10434 192.168.150.89/53435 query[A] app.snapchat.com from 192.168.150.89
Apr 27 09:31:41 dnsmasq[797]: 10434 192.168.150.89/53435 forwarded app.snapchat.com to 127.0.2.2
Apr 27 09:31:41 dnsmasq[797]: 10434 192.168.150.89/53435 validation result is INSECURE
Apr 27 09:31:41 dnsmasq[797]: 10434 192.168.150.89/53435 reply app.snapchat.com is <CNAME>
Apr 27 09:31:41 dnsmasq[797]: 10435 192.168.150.89/56737 query[A] sc-analytics.appspot.com from 192.168.150.89
Apr 27 09:31:41 dnsmasq[797]: 10435 192.168.150.89/56737 /etc/pihole/gravity.list sc-analytics.appspot.com is 192.168.150.48
Apr 27 09:31:41 dnsmasq[797]: 10436 192.168.150.89/51485 query[A] app-analytics.snapchat.com from 192.168.150.89
Apr 27 09:31:41 dnsmasq[797]: 10436 192.168.150.89/51485 forwarded app-analytics.snapchat.com to 127.0.2.2
Apr 27 09:31:41 dnsmasq[797]: 10436 192.168.150.89/51485 validation app-analytics.snapchat.com is BOGUS
Apr 27 09:31:41 dnsmasq[797]: 10437 192.168.150.89/63203 query[A] am-prod.sc-jpl.com from 192.168.150.89.
Apr 27 09:31:41 dnsmasq[797]: 10437 192.168.150.89/63203 forwarded am-prod.sc-jpl.com to 127.0.2.2
Apr 27 09:31:41 dnsmasq[797]: 10437 192.168.150.89/63203 validation result is INSECURE
Apr 27 09:31:47 dnsmasq[797]: 10439 192.168.150.90/60997 query[A] gateway.fe.apple-dns.net from 192.168.150.90
Apr 27 09:31:47 dnsmasq[797]: 10439 192.168.150.90/60997 forwarded gateway.fe.apple-dns.net to 127.0.2.2
Apr 27 09:31:47 dnsmasq[797]: 10439 192.168.150.90/60997 validation gateway.fe.apple-dns.net is BOGUS`

jpgpi250 · April 27, 2018, 8:15am

Here is what I do to check if things have improved after a change:

sudo /opt/dnscrypt-proxy/dnscrypt-proxy -service restart
sudo service dnsmasq restart
sudo apt-get update

The result:

Apr 27 10:02:49 dnsmasq[20851]: started, version 2.80test2 cachesize 65535
Apr 27 10:02:49 dnsmasq[20851]: compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect inotify
Apr 27 10:02:49 dnsmasq[20851]: DNSSEC validation enabled
Apr 27 10:02:49 dnsmasq[20851]: warning: ignoring resolv-file flag because no-resolv is set
Apr 27 10:02:49 dnsmasq[20851]: using nameserver 127.10.10.1#5551
Apr 27 10:02:49 dnsmasq[20851]: using local addresses only for domain localdomain
Apr 27 10:03:00 dnsmasq[20851]: read /etc/hosts - 14 addresses
Apr 27 10:03:00 dnsmasq[20851]: read /etc/pihole/local.list - 2 addresses
Apr 27 10:03:00 dnsmasq[20851]: failed to load names from /etc/pihole/black.list: No such file or directory
Apr 27 10:03:07 dnsmasq[20851]: bad name at /etc/pihole/gravity.list line 471222
Apr 27 10:03:09 dnsmasq[20851]: read /etc/pihole/gravity.list - 601177 addresses
Apr 27 10:03:09 dnsmasq[20851]: read /etc/localdns.list - 39 addresses
Apr 27 10:03:09 dnsmasq[20851]: 1 127.0.0.1/37576 query[SRV] _http._tcp.archive.raspberrypi.org from 127.0.0.1
Apr 27 10:03:09 dnsmasq[20851]: 1 127.0.0.1/37576 forwarded _http._tcp.archive.raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: 2 127.0.0.1/46107 query[SRV] _http._tcp.raspbian.raspberrypi.org from 127.0.0.1
Apr 27 10:03:09 dnsmasq[20851]: 2 127.0.0.1/46107 forwarded _http._tcp.raspbian.raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20928]: 3 127.0.0.1/55314 query[SRV] _http._tcp.raspbian.raspberrypi.org from 127.0.0.1
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37576 dnssec-query[DS] org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37576 dnssec-query[DNSKEY] . to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37576 reply . is DNSKEY keytag 19036, algo 8
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37576 reply . is DNSKEY keytag 39570, algo 8
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37576 reply . is DNSKEY keytag 20326, algo 8
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37576 reply org is DS keytag 9795, algo 7, digest 1
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37576 reply org is DS keytag 9795, algo 7, digest 2
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37576 dnssec-query[DS] raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20928]: 3 127.0.0.1/55314 forwarded _http._tcp.raspbian.raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 dnssec-query[DS] org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37576 dnssec-query[DNSKEY] org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 dnssec-query[DNSKEY] . to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20929]: 103 127.0.0.1/55322 query[SRV] _http._tcp.archive.raspberrypi.org from 127.0.0.1
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply . is DNSKEY keytag 20326, algo 8
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply . is DNSKEY keytag 39570, algo 8
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply . is DNSKEY keytag 19036, algo 8
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply org is DS keytag 9795, algo 7, digest 2
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply org is DS keytag 9795, algo 7, digest 1
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 dnssec-query[DS] raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20929]: 103 127.0.0.1/55322 forwarded _http._tcp.archive.raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 dnssec-query[DS] raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 dnssec-query[DNSKEY] org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 dnssec-query[DNSKEY] org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply org is DNSKEY keytag 17883, algo 7
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply org is DNSKEY keytag 1862, algo 7
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply org is DNSKEY keytag 9795, algo 7
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply org is DNSKEY keytag 6368, algo 7
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply raspberrypi.org is DS keytag 49776, algo 10, digest 2
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply raspberrypi.org is DS keytag 24957, algo 10, digest 2
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 dnssec-query[DNSKEY] raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 reply org is DNSKEY keytag 9795, algo 7
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 reply org is DNSKEY keytag 17883, algo 7
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 reply org is DNSKEY keytag 1862, algo 7
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 reply org is DNSKEY keytag 6368, algo 7
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 reply raspberrypi.org is DS keytag 24957, algo 10, digest 2
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 reply raspberrypi.org is DS keytag 49776, algo 10, digest 2
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 dnssec-query[DNSKEY] raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply raspberrypi.org is DNSKEY keytag 33908, algo 10
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply raspberrypi.org is DNSKEY keytag 49776, algo 10
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply raspberrypi.org is DNSKEY keytag 24957, algo 10
Apr 27 10:03:09 dnsmasq[20928]: * 127.0.0.1/55314 reply raspberrypi.org is DNSKEY keytag 23675, algo 10
Apr 27 10:03:09 dnsmasq[20928]: 3 127.0.0.1/55314 validation result is SECURE
Apr 27 10:03:09 dnsmasq[20851]: 203 127.0.0.1/37852 query[A] raspbian.raspberrypi.org from 127.0.0.1
Apr 27 10:03:09 dnsmasq[20851]: 203 127.0.0.1/37852 forwarded raspbian.raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: 204 127.0.0.1/37852 query[AAAA] raspbian.raspberrypi.org from 127.0.0.1
Apr 27 10:03:09 dnsmasq[20851]: 204 127.0.0.1/37852 forwarded raspbian.raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37852 dnssec-query[DS] raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 reply raspberrypi.org is DNSKEY keytag 49776, algo 10
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 reply raspberrypi.org is DNSKEY keytag 23675, algo 10
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 reply raspberrypi.org is DNSKEY keytag 33908, algo 10
Apr 27 10:03:09 dnsmasq[20929]: * 127.0.0.1/55322 reply raspberrypi.org is DNSKEY keytag 24957, algo 10
Apr 27 10:03:09 dnsmasq[20929]: 103 127.0.0.1/55322 validation result is SECURE
Apr 27 10:03:09 dnsmasq[20851]: 205 127.0.0.1/33024 query[A] archive.raspberrypi.org from 127.0.0.1
Apr 27 10:03:09 dnsmasq[20851]: 205 127.0.0.1/33024 forwarded archive.raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: 206 127.0.0.1/33024 query[AAAA] archive.raspberrypi.org from 127.0.0.1
Apr 27 10:03:09 dnsmasq[20851]: 206 127.0.0.1/33024 forwarded archive.raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/33024 dnssec-query[DS] raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37852 dnssec-query[DNSKEY] org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/33024 dnssec-query[DNSKEY] org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20931]: 207 127.0.0.1/55338 query[A] archive.raspberrypi.org from 127.0.0.1
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37852 dnssec-query[DS] raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: nameserver 127.10.10.1 refused to do a recursive query
Apr 27 10:03:09 dnsmasq[20851]: nameserver 127.10.10.1 refused to do a recursive query
Apr 27 10:03:09 dnsmasq[20932]: 307 127.0.0.1/55342 query[A] raspbian.raspberrypi.org from 127.0.0.1
Apr 27 10:03:09 dnsmasq[20851]: * 127.0.0.1/37852 dnssec-query[DNSKEY] org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20851]: 203 127.0.0.1/37852 reply raspbian.raspberrypi.org is <CNAME>
Apr 27 10:03:09 dnsmasq[20851]: 203 127.0.0.1/37852 reply mirrordirector.raspbian.org is 93.93.128.193
Apr 27 10:03:09 dnsmasq[20931]: 207 127.0.0.1/55338 forwarded archive.raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20931]: * 127.0.0.1/55338 dnssec-query[DS] raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20932]: 307 127.0.0.1/55342 forwarded raspbian.raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20932]: * 127.0.0.1/55342 dnssec-query[DS] raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20931]: * 127.0.0.1/55338 dnssec-query[DNSKEY] org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20931]: * 127.0.0.1/55338 reply org is DNSKEY keytag 9795, algo 7
Apr 27 10:03:09 dnsmasq[20931]: * 127.0.0.1/55338 reply org is DNSKEY keytag 17883, algo 7
Apr 27 10:03:09 dnsmasq[20931]: * 127.0.0.1/55338 reply org is DNSKEY keytag 1862, algo 7
Apr 27 10:03:09 dnsmasq[20931]: * 127.0.0.1/55338 reply org is DNSKEY keytag 6368, algo 7
Apr 27 10:03:09 dnsmasq[20931]: * 127.0.0.1/55338 reply raspberrypi.org is DS keytag 49776, algo 10, digest 2
Apr 27 10:03:09 dnsmasq[20931]: * 127.0.0.1/55338 reply raspberrypi.org is DS keytag 24957, algo 10, digest 2
Apr 27 10:03:09 dnsmasq[20931]: * 127.0.0.1/55338 dnssec-query[DNSKEY] raspberrypi.org to 127.10.10.1
Apr 27 10:03:09 dnsmasq[20932]: * 127.0.0.1/55342 dnssec-query[DNSKEY] org to 127.10.10.1
Apr 27 10:03:10 dnsmasq[20931]: * 127.0.0.1/55338 reply raspberrypi.org is DNSKEY keytag 49776, algo 10
Apr 27 10:03:10 dnsmasq[20931]: * 127.0.0.1/55338 reply raspberrypi.org is DNSKEY keytag 33908, algo 10
Apr 27 10:03:10 dnsmasq[20931]: * 127.0.0.1/55338 reply raspberrypi.org is DNSKEY keytag 23675, algo 10
Apr 27 10:03:10 dnsmasq[20931]: * 127.0.0.1/55338 reply raspberrypi.org is DNSKEY keytag 24957, algo 10
Apr 27 10:03:10 dnsmasq[20931]: 207 127.0.0.1/55338 validation result is SECURE
Apr 27 10:03:10 dnsmasq[20931]: 207 127.0.0.1/55338 reply archive.raspberrypi.org is <CNAME>
Apr 27 10:03:10 dnsmasq[20931]: 207 127.0.0.1/55338 reply lb.raspberrypi.org is 93.93.130.39
Apr 27 10:03:10 dnsmasq[20931]: 207 127.0.0.1/55338 reply lb.raspberrypi.org is 93.93.130.104
Apr 27 10:03:10 dnsmasq[20931]: 207 127.0.0.1/55338 reply lb.raspberrypi.org is 93.93.130.214
Apr 27 10:03:10 dnsmasq[20931]: 207 127.0.0.1/55338 reply lb.raspberrypi.org is 93.93.135.188
Apr 27 10:03:10 dnsmasq[20931]: 207 127.0.0.1/55338 reply lb.raspberrypi.org is 46.235.227.11
Apr 27 10:03:10 dnsmasq[20931]: 207 127.0.0.1/55338 reply lb.raspberrypi.org is 93.93.128.133
Apr 27 10:03:10 dnsmasq[20931]: 207 127.0.0.1/55338 reply lb.raspberrypi.org is 93.93.128.211
Apr 27 10:03:10 dnsmasq[20931]: 207 127.0.0.1/55338 reply lb.raspberrypi.org is 93.93.128.230
Apr 27 10:03:10 dnsmasq[20931]: 208 127.0.0.1/55338 query[AAAA] archive.raspberrypi.org from 127.0.0.1
Apr 27 10:03:10 dnsmasq[20931]: 208 127.0.0.1/55338 cached archive.raspberrypi.org is <CNAME>
Apr 27 10:03:10 dnsmasq[20931]: 208 127.0.0.1/55338 forwarded archive.raspberrypi.org to 127.10.10.1
Apr 27 10:03:10 dnsmasq[20931]: * 127.0.0.1/55338 dnssec-query[DS] archive.raspberrypi.org to 127.10.10.1
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply org is DNSKEY keytag 6368, algo 7
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply org is DNSKEY keytag 1862, algo 7
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply org is DNSKEY keytag 9795, algo 7
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply org is DNSKEY keytag 17883, algo 7
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply raspberrypi.org is DS keytag 49776, algo 10, digest 2
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply raspberrypi.org is DS keytag 24957, algo 10, digest 2
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 dnssec-query[DNSKEY] raspberrypi.org to 127.10.10.1
Apr 27 10:03:10 dnsmasq[20931]: * 127.0.0.1/55338 reply archive.raspberrypi.org is no DS
Apr 27 10:03:10 dnsmasq[20931]: 208 127.0.0.1/55338 validation archive.raspberrypi.org is BOGUS
Apr 27 10:03:10 dnsmasq[20931]: nameserver 127.10.10.1 refused to do a recursive query
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply raspberrypi.org is DNSKEY keytag 33908, algo 10
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply raspberrypi.org is DNSKEY keytag 24957, algo 10
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply raspberrypi.org is DNSKEY keytag 23675, algo 10
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply raspberrypi.org is DNSKEY keytag 49776, algo 10
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 dnssec-query[DS] raspbian.org to 127.10.10.1
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply raspbian.org is no DS
Apr 27 10:03:10 dnsmasq[20932]: 307 127.0.0.1/55342 validation result is INSECURE
Apr 27 10:03:10 dnsmasq[20932]: 307 127.0.0.1/55342 reply raspbian.raspberrypi.org is <CNAME>
Apr 27 10:03:10 dnsmasq[20932]: 307 127.0.0.1/55342 reply mirrordirector.raspbian.org is 93.93.128.193
Apr 27 10:03:10 dnsmasq[20932]: 308 127.0.0.1/55342 query[AAAA] raspbian.raspberrypi.org from 127.0.0.1
Apr 27 10:03:10 dnsmasq[20932]: 308 127.0.0.1/55342 cached raspbian.raspberrypi.org is <CNAME>
Apr 27 10:03:10 dnsmasq[20932]: 308 127.0.0.1/55342 forwarded raspbian.raspberrypi.org to 127.10.10.1
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 dnssec-query[DS] raspbian.raspberrypi.org to 127.10.10.1
Apr 27 10:03:10 dnsmasq[20932]: Insecure DS reply received, do upstream DNS servers support DNSSEC?
Apr 27 10:03:10 dnsmasq[20932]: * 127.0.0.1/55342 reply raspbian.raspberrypi.org is BOGUS DS
Apr 27 10:03:10 dnsmasq[20932]: 308 127.0.0.1/55342 validation raspbian.raspberrypi.org is BOGUS
Apr 27 10:03:10 dnsmasq[20932]: nameserver 127.10.10.1 refused to do a recursive query

Sometimes, I even get (on the pi console):

sudo apt-get update
Err:1 http://raspbian.raspberrypi.org/raspbian stretch InRelease
  Temporary failure resolving 'raspbian.raspberrypi.org'
Hit:2 http://archive.raspberrypi.org/debian stretch InRelease

Reading package lists... Done
W: Failed to fetch http://raspbian.raspberrypi.org/raspbian/dists/stretch/InRelease  Temporary failure resolving 'raspbian.raspberrypi.org'
W: Some index files failed to download. They have been ignored, or old ones used instead.

In the mean time, I'm still continuing my conversation with the dnsmasq developer.

jpgpi250 · April 27, 2018, 9:28am

Something strange I noticed:

sudo service dnsmasq restart
dig +dnssec ds archive.raspberrypi.org

the result for archive.raspberry.org is SECURE

sudo service dnsmasq restart
sudo apt-get update

the result for archive.raspberry.org is BOGUS

Anybody able to confirm this result?

jpgpi250 · April 27, 2018, 1:46pm

A part of the problem has been identified.
As mentioned earlier, I'm running dnsmasq2.80test2 and dnscrypt-proxy 2.0.11

I only have ipv4, my provider refuses to exchange my docsis 2 (ipv4 only) modem for a docsis 3 modem, so ...

In the wiki, section 'making things go faster', there is a setting 'block_ipv6 = false'. The wiki says to change it, so I did.
WRONG!!!
If you only have IPv4 and you're using dnsmasq + dnscrypt-proxy V2 + DNSSEC, don't change that setting!!!!

justification (from the dnsmasq developer):
'quote'
Looking at your logs, this is responsible for some of the BOGUS validations, if dnscrypt-proxy gives a synthetic reply to a query in a signed domain, that will fail DNSSEC validation, obviously.
If your network doesn't support IPv6, chances are that your applications are still constantly trying to resolve IPv6 addresses, causing unnecessary slowdowns.
This causes the proxy to immediately reply to IPv6 requests, without having to send a useless request to upstream resolvers, and having to wait for a response.
This uses a plugin that requires dnscrypt-proxy to be compiled with the ldns library.
'/quote'

I changed the setting back to it's default ('block_ipv6 = false') and some of the BOGUS errors have disappeared. In the item above (something strange I noticed), 'archive.raspberry.org' came up as BOGUS, when using 'sudo apt-get update'.
That problem is now solved by reverting to the default dnscrypt-proxy setting, however, still a long way to go for other false BOGUS errors...

To be continued.

DL6ER · April 27, 2018, 3:29pm

DNSSEC problems are something we can really only reduce by always offering the latest (stable) version of dnsmasq through FTLDNS. It is clear to us that the DNSSEC implementation is dnsmasq is far from being completely bug free. This is also the reason why we never recommend to enable DNSSEC for Pi-hole's in any productivity level installation of Pi-hole as you have to expect catches and non-resolving domains.

I know that this is not particularly helpful, but there are few things we really can do about it. Furthermore, I'm not convinced that DNSSEC is really "worth it" (at least not right now) as even the majority of the big global players aren't investing in this (they always come as INSECURE).

I have not tried the archive.raspberrypi.org issue myself, but I seem to remember that apt update frequently queries mirrordirector.raspbian.org so it may be that there is a conflict caused by this.

I can understand this, but I can also assure you that with respect to DNSSEC, nothing has changed since the very early days of FTLDNS.

I move this from the Help category over to General as there isn't much we can do about it right now except from updating FTLDNS's internal resolver to the next dnsmasq version once this is considered stable.

jpgpi250 · April 27, 2018, 3:57pm

DL6ER, nothing personal, absolutely NOT, apologies if I offend you.
But this is a bad statement that doesn't help furthering DNSSEC.
Tech people should grab every opportunity to request (tech) sites to create DNSSEC records.
discourse.pi-hole.net should of had DNSSEC records the day pihole first supported it.

Apr 27 17:41:45 dnsmasq[3869]: 920 192.168.2.227/56834 query[A] discourse.pi-hole.net from 192.168.2.227
Apr 27 17:41:45 dnsmasq[3869]: 920 192.168.2.227/56834 forwarded discourse.pi-hole.net to 127.10.10.1
Apr 27 17:41:45 dnsmasq[3869]: 920 192.168.2.227/56834 validation result is INSECURE

If the DNS administrators keep being annoyed with requests to support DNSSEC, sooner or later at least some of them will create them...

Just my two cents...

DL6ER · April 27, 2018, 5:23pm

No offense taken. However, you misinterpreted my statement.

I said

(ignore the part with the global players for now). I didn't intend to say DNSSEC will never worth it, but, in fact, it isn't today. Users enable DNSSEC and start experiencing problems. If they are not able to properly analyze the root of the issues, the only thing they get is a screwed up Internet experience.

This statement is a pure support-oriented statement (talking to users on any level of technical experience and knowledge of DNS). I, in fact, run a recursive DNS server on the same device Pi-hole is running on (I think you are aware of the wiki article I wrote). Here, I fully use DNSSEC (I even use harden-dnssec-stripped), but then I' also aware of what I need to do in case some domains don't resolve erroneously. I just cannot be bothered to recommend switching on DNSSEC right now for an arbitrary user (the benefit is low as long as many domains are anyhow INSECURE).

Oh, we actually had DNSSEC for some time, but it was causing issues all over the place. When you are INSECURE, then users can visit your page just fine. However, as soon as you configure DNSSEC for your page, each very minor perturbation will immediately render your page BOGUS and users with DNSSEC validation won't be able to use Discourse any more.

If it would only be so simple ... we tried hard for some time and eventually came to the conclusion that it is not worth the trouble it was causing. @DanSchaper would be able to give you more details on our implementation efforts on the Pi-hole domains and why it got removed again.

DanSchaper · April 27, 2018, 6:27pm

We did have DNSSEC enabled, but rolling DS keys to the registrars that are not the same as our DNS providers is not the most elegant solution. I already spend more than full time hours supporting the infrastructure that makes everything work, and I don't have time to shuffle records between providers, between registrars and have to worry that all of our TLDs will be non-resolvable if something goes wrong somewhere, either with my process or with one of our providers. Maybe when the process matures it will be revisited, but for now it's too much work, too prone to failures and too manually intensive to implement.

DanSchaper · April 27, 2018, 6:29pm

Also, I should mention that many registrars and DNS providers do not even offer the option of DNSSEC because of the high support load it creates. It's not like a bad record can time out with a cache purge or a TTL time out, you break the chain and it's game over for a long time. We can't risk that.

gecko · April 28, 2018, 8:18am

I'm not sure the best solution is to simply disable DNSSEC. I realise it comes with its technical difficulties and bugs, but at the moment it's the best we've got for some kind of DNS security against DNS hijacking (along with DNSCrypt/DNS-over-TLS/DoH), and until the industry comes up with something better, we should be encouraged to use it. Only with more people using it can we hope for it to either be improved, or a better replacement for it to be made.

I had a thought though. If it is, as suspected, due to issues between dnsmasq and DNSSEC, would it better for me to disable DNSSEC in the pihole interface (and thus on the FTLDNS/dnsmasq side of things), and only enable it on the stubby resolver? Of course, I wouldn't then be able to tell from the pihole UI what the issue was if there was a problem with DNSSEC resolution, but I assume that any result that was genuinely BOGUS wouldn't resolve on stubby, and so wouldn't be passed on to pihole, and thus I gain the same protection. Or would that not work?

mibere · April 28, 2018, 8:34am

It's ridiculous and embarrassing, DNSSEC is available since 2010/2011...and still has or leads to difficulties.

(General statement, not related to Pi-hole.)

DL6ER · April 28, 2018, 9:15am

Yes, see also this comment I made:

I don't know anything about the stubby resolver, however, if it works as we'd expect, then it wouldn't give you a reply for a BOGUS domain and hence Pi-hole wouldn't be able to give your clients a reply and everything would be fine.

In my configuration, I can verify this by:

$ dig www.dnssec-failed.org 

; <<>> DiG 9.10.3-P4-Ubuntu <<>> www.dnssec-failed.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 19178
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.dnssec-failed.org.		IN	A

;; Query time: 50 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Apr 28 11:01:17 CEST 2018
;; MSG SIZE  rcvd: 50

My Pi-hole isn't aware of DNSSEC at all and it is just my unbound resolver which is protecting me here:

[1524906872] unbound[31278:0] info: response for dnssec-failed.org. DNSKEY IN
[1524906872] unbound[31278:0] info: reply from <dnssec-failed.org.> 68.87.68.244#53
[1524906872] unbound[31278:0] info: query response was ANSWER
[1524906872] unbound[31278:0] info: Did not match a DS to a DNSKEY, thus bogus.