The issue I am facing: Since somewhere last night my Pi-hole has stopped working. All queries not in cache are returning Blocked (external, NXRA). This would indicate that the upstream DNS server is blocking the domain. I find it hard to believe that common domains such as google.com, pool.ntp.org, etc would be blocked by my upstream DNS dns0.net. Gravity lists also won’t update.
Troubleshooting steps: I’ve switched my upstream DNS to Quad9 in case the DNS provider had issues. I restarted the pi-hole DNS resolver. I’ve restarted the entire system. I’ve now bypassed the pi-hole while still using dns0.net and DNS is working normally.
Details about my system: Raspberry Pi 4 - Core v6.1.4 | FTL v6.2.3 | Web interface v6.2.1
What I have changed since installing Pi-hole: No changes since it has stopped working. There are 2 custom DNS upstream servers, a few local DNS records for accessing machines on the local network, DNS interface is set to “allow all origins”, pi-hole is not a DHCP server.
Please upload a debug log and post just the token URL that is generated after the log is uploaded by running the following command from the Pi-hole host terminal:
Your router is configured to advertise itself as DNS server, so your devices won't use Pi-hole, unless you set Pi-hole IP as DNS server directly on each device.
I only had dns0.eu configured initially, I added openDNS after I was experiencing issues. I’ve checked the log on the day that it occurred, it looks like some queries aren’t being forwarded or resolved from cache, but immediate get the blocked response:
Sep 28 00:47:27 dnsmasq[145067]: query[A] www.youtube.com from 192.168.1.123
Sep 28 00:47:27 dnsmasq[145067]: blocked upstream with NXRA address www.youtube.com is 0.0.0.0
Sep 28 00:47:28 dnsmasq[145067]: query[A] www.google.com from 192.168.1.123
Sep 28 00:47:28 dnsmasq[145067]: blocked upstream with NXRA address www.google.com is 0.0.0.0
Sep 28 00:47:41 dnsmasq[145067]: query[A] homegraph.googleapis.com from 192.168.1.140
Sep 28 00:47:41 dnsmasq[145067]: blocked upstream with NXRA address homegraph.googleapis.com is 0.0.0.0
Sep 28 00:47:41 dnsmasq[145067]: query[AAAA] homegraph.googleapis.com from 192.168.1.140
Sep 28 00:47:41 dnsmasq[145067]: blocked upstream with NXRA address homegraph.googleapis.com is ::
Sep 28 00:48:12 dnsmasq[145067]: query[A] clients3.google.com from 192.168.1.123
Sep 28 00:48:12 dnsmasq[145067]: blocked upstream with NXRA address clients3.google.com is 0.0.0.0
I will stop using dns0.eu as my primary resolver and see if the issue re-occurs.
I haven't seen such a line before.
It's usually either NXRA or 0.0.0.0, and the latter would show up as Blocked (external, NULL):
Oct 1 11:48:33 dnsmasq[53]: query[A] 123.ywxww.net from 192.168.1.201
Oct 1 11:48:33 dnsmasq[53]: forwarded 123.ywxww.net to 116.203.32.217
Oct 1 11:48:33 dnsmasq[53]: reply 123.ywxww.net is blocked due to upstream response (answer)
Oct 1 11:48:33 dnsmasq[53]: <not set> 123.ywxww.net is 0.0.0.0
Oct 1 12:01:12 dnsmasq[53]: query[A] 123.ywxww.net from 192.168.1.201
Oct 1 12:01:12 dnsmasq[53]: forwarded 123.ywxww.net to 9.9.9.9
Oct 1 12:01:12 dnsmasq[53]: reply 123.ywxww.net is blocked due to upstream response (header)
Oct 1 12:01:12 dnsmasq[53]: blocked upstream with NXDOMAIN + no RA 123.ywxww.net is 0.0.0.0`
I'd also expected to see a forward in your logs.
Are your quoted log lines complete, or did you omit some lines?
Ok, it appears that line shows up for successive DNS requests for externally blocked domains:
Oct 1 12:01:12 dnsmasq[53]: query[A] 123.ywxww.net from 192.168.1.201
Oct 1 12:01:12 dnsmasq[53]: forwarded 123.ywxww.net to 9.9.9.9
Oct 1 12:01:12 dnsmasq[53]: reply 123.ywxww.net is blocked due to upstream response (header)
Oct 1 12:01:12 dnsmasq[53]: blocked upstream with NXDOMAIN + no RA 123.ywxww.net is 0.0.0.0
Oct 1 12:02:28 dnsmasq[53]: query[A] 123.ywxww.net from 192.168.1.201
Oct 1 12:02:28 dnsmasq[53]: blocked upstream with NXRA address 123.ywxww.net is 0.0.0.0
That would indicate that Pi-hole is caching that blocked reply, so it has blocked it without forwarding.
To find the upstream that originally blocked the domain, you'd had to look for the first requests. I guess it's that upstream that would have blocked the original first request, and Pi-hole may still be caching that blocked reply.
This is still surprising, as my Pi-hole has answered the inital request with a TTL of 2 seconds - even if its cached, Pi-hole's cache optimiser should have forwarded it upstream while providing a cached-stale reply.
While the block would be caused by your upstream, it looks like Pi-hole may artificially prolong that block by caching it longer than expected.
Could you take a look here, @DL6ER ?
EDIT: It seems like Pi-hole's cache optimiser may not be treating replies that are blocked by upstreams in the same way as regular upstream replies.
Ok, so I looked up resolution for app.tado.com and it is dns0.eu which blocks it initially:
Sep 28 01:12:45 dnsmasq[145067]: query[A] app.tado.com from 192.168.1.94
Sep 28 01:12:45 dnsmasq[145067]: forwarded app.tado.com to 193.110.81.0
Sep 28 01:12:47 dnsmasq[145067]: reply app.tado.com is blocked due to upstream response (header)
Sep 28 01:12:47 dnsmasq[145067]: blocked upstream with NXDOMAIN + no RA app.tado.com is 0.0.0.0
After that subsequent queries are no longer forwarded.
Usually, upstream servers shouldn't reply with NXDOMAIN and no RA (= recursion available) bit set. I am not sure which RFC to quote for this right now, however, recursive DNS server replying that they cannot offer recursion doesn't make sense.
The issue is that the upstream is replying with NXDOMAIN where it shouldn't and, thus, seems severely broken. If it replies to NXDOMAIN to any domain, Pi-hole will cache this (configurable through dns.cache.upstreamBlockedTTL, defaulting to 24 hours). The problem is made even larger due to the fact that NXDOMAIN does not only mean this domain does not exist but also none of its subdomains exist. This means that your Pi-hole will not even try to resolve them as the upstream says that this all does not exist.