Please add logging to yourunbound and show us the full log output (in /var/log/unbound.log) from starting unbound to after you made the dig call. This should give us a hint what is amiss here.
This might sound weird, but I don't think the unbound service is using my config file.
I just did what was in that link for adding debugging to unbound, restarted the service, ran the same dig command I have been, and the log file is empty.
Here's what I did:
Added the following to my /etc/unbound/unbound.conf.d/pi-hole.conf file at the top of the server section:
● unbound.service - Unbound DNS server
Loaded: loaded (/lib/systemd/system/unbound.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2022-12-30 23:12:10 CST; 38min ago
Docs: man:unbound(8)
Process: 23992 ExecStartPre=/usr/lib/unbound/package-helper chroot_setup (code=exited, status=0/SUCCESS)
Process: 23995 ExecStartPre=/usr/lib/unbound/package-helper root_trust_anchor_update (code=exited, status=0/SUCCESS)
Main PID: 23998 (unbound)
Tasks: 1 (limit: 1624)
CPU: 192ms
CGroup: /system.slice/unbound.service
└─23998 /usr/sbin/unbound -d -p
Dec 30 23:19:23 raspberrypi unbound[23998]: [1672463963] unbound[23998:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
Dec 30 23:19:23 raspberrypi unbound[23998]: [1672463963] unbound[23998:0] info: generate keytag query _ta-4f66. NULL IN
Dec 30 23:19:23 raspberrypi unbound[23998]: [1672463963] unbound[23998:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
Dec 30 23:19:23 raspberrypi unbound[23998]: [1672463963] unbound[23998:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
Dec 30 23:19:23 raspberrypi unbound[23998]: [1672463963] unbound[23998:0] info: generate keytag query _ta-4f66. NULL IN
Dec 30 23:19:24 raspberrypi unbound[23998]: [1672463964] unbound[23998:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
Dec 30 23:19:24 raspberrypi unbound[23998]: [1672463964] unbound[23998:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
Dec 30 23:19:24 raspberrypi unbound[23998]: [1672463964] unbound[23998:0] info: generate keytag query _ta-4f66. NULL IN
Dec 30 23:19:24 raspberrypi unbound[23998]: [1672463964] unbound[23998:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
Dec 30 23:19:24 raspberrypi unbound[23998]: [1672463964] unbound[23998:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
to the end (or, /tmp/unbound.log whatever you want to use - just make sure the value is the same in the unbound config). Then reload AppArmor using
sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.unbound
sudo service unbound restart
and then we shall see if this helped... If it did, we will add this to the documentation, too. AppArmor might simply be deployed much wider by now than it was when we initially wrote the unbound guide.
[1672468980] unbound[29165:0] notice: init module 0: subnet
[1672468980] unbound[29165:0] notice: init module 1: validator
[1672468980] unbound[29165:0] notice: init module 2: iterator
[1672468980] unbound[29165:0] info: start of service (unbound 1.13.1).
[1672469056] unbound[29165:0] info: generate keytag query _ta-4f66. NULL IN
[1672469056] unbound[29165:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672469056] unbound[29165:0] info: generate keytag query _ta-4f66. NULL IN
[1672469056] unbound[29165:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672469056] unbound[29165:0] info: generate keytag query _ta-4f66. NULL IN
[1672469056] unbound[29165:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672469056] unbound[29165:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672469056] unbound[29165:0] info: generate keytag query _ta-4f66. NULL IN
[1672469056] unbound[29165:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672469056] unbound[29165:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672469056] unbound[29165:0] info: generate keytag query _ta-4f66. NULL IN
[1672469056] unbound[29165:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672469056] unbound[29165:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672469056] unbound[29165:0] info: generate keytag query _ta-4f66. NULL IN
[1672469056] unbound[29165:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672469056] unbound[29165:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
There's is one other file in /etc/unbound/unbound.conf.d/. It's called root-auto-trust-anchor-file.conf and here's what's in it:
server:
# The following line will configure unbound to perform cryptographic
# DNSSEC validation using the root trust anchor.
auto-trust-anchor-file: "/var/lib/unbound/root.key"
That's fine, I do have the same file, it is installed alongside unbound by the operating system. I just realized that I have the root-hints: line commented out in my pi-hole.conf as it is already contained in the other file.
I increased the verbosity to 2 and this is what I get now:
[1672470141] unbound[30234:0] notice: init module 2: iterator
[1672470142] unbound[30234:0] info: start of service (unbound 1.13.1).
[1672470164] unbound[30234:0] info: resolving pi-hole.net. A IN
[1672470164] unbound[30234:0] info: priming . IN NS
[1672470164] unbound[30234:0] info: response for . NS IN
[1672470164] unbound[30234:0] info: reply from <.> 192.5.5.241#53
[1672470164] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470164] unbound[30234:0] info: mark as REC_LAME
[1672470164] unbound[30234:0] info: response for . NS IN
[1672470164] unbound[30234:0] info: reply from <.> 199.7.83.42#53
[1672470164] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470164] unbound[30234:0] info: mark as REC_LAME
[1672470164] unbound[30234:0] info: response for . NS IN
[1672470164] unbound[30234:0] info: reply from <.> 198.41.0.4#53
[1672470164] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470164] unbound[30234:0] info: mark as REC_LAME
[1672470164] unbound[30234:0] info: response for . NS IN
[1672470164] unbound[30234:0] info: reply from <.> 202.12.27.33#53
[1672470164] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470164] unbound[30234:0] info: mark as REC_LAME
[1672470164] unbound[30234:0] info: response for . NS IN
[1672470164] unbound[30234:0] info: reply from <.> 193.0.14.129#53
[1672470164] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470164] unbound[30234:0] info: mark as REC_LAME
[1672470164] unbound[30234:0] info: response for . NS IN
[1672470164] unbound[30234:0] info: reply from <.> 199.9.14.201#53
[1672470164] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470164] unbound[30234:0] info: mark as REC_LAME
[1672470164] unbound[30234:0] info: response for . NS IN
[1672470164] unbound[30234:0] info: reply from <.> 192.203.230.10#53
[1672470164] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470164] unbound[30234:0] info: mark as REC_LAME
[1672470165] unbound[30234:0] info: response for . NS IN
[1672470165] unbound[30234:0] info: reply from <.> 192.33.4.12#53
[1672470165] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470165] unbound[30234:0] info: mark as REC_LAME
[1672470165] unbound[30234:0] info: response for . NS IN
[1672470165] unbound[30234:0] info: reply from <.> 198.97.190.53#53
[1672470165] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470165] unbound[30234:0] info: mark as REC_LAME
[1672470165] unbound[30234:0] info: response for . NS IN
[1672470165] unbound[30234:0] info: reply from <.> 192.112.36.4#53
[1672470165] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470165] unbound[30234:0] info: mark as REC_LAME
[1672470165] unbound[30234:0] info: response for . NS IN
[1672470165] unbound[30234:0] info: reply from <.> 192.36.148.17#53
[1672470165] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470165] unbound[30234:0] info: mark as REC_LAME
[1672470165] unbound[30234:0] info: response for . NS IN
[1672470165] unbound[30234:0] info: reply from <.> 199.7.91.13#53
[1672470165] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470165] unbound[30234:0] info: mark as REC_LAME
[1672470165] unbound[30234:0] info: response for . NS IN
[1672470165] unbound[30234:0] info: reply from <.> 192.58.128.30#53
[1672470165] unbound[30234:0] info: query response REC_LAME: recursive but not authoritative server
[1672470165] unbound[30234:0] info: mark as REC_LAME
[1672470165] unbound[30234:0] info: response for . NS IN
[1672470165] unbound[30234:0] info: reply from <.> 198.97.190.53#53
[1672470165] unbound[30234:0] info: query response was ANSWER
[1672470165] unbound[30234:0] info: priming successful for . NS IN
[1672470165] unbound[30234:0] info: resolving a.root-servers.net. A IN
[1672470165] unbound[30234:0] info: response for a.root-servers.net. A IN
[1672470165] unbound[30234:0] info: reply from <.> 192.5.5.241#53
[1672470165] unbound[30234:0] info: query response was nodata ANSWER
[1672470165] unbound[30234:0] info: response for a.root-servers.net. A IN
[1672470165] unbound[30234:0] info: reply from <.> 192.36.148.17#53
[1672470165] unbound[30234:0] info: query response was nodata ANSWER
[1672470165] unbound[30234:0] info: response for a.root-servers.net. A IN
[1672470165] unbound[30234:0] info: reply from <.> 192.203.230.10#53
[1672470165] unbound[30234:0] info: query response was ANSWER
[1672470165] unbound[30234:0] info: response for pi-hole.net. A IN
[1672470165] unbound[30234:0] info: reply from <.> 199.7.83.42#53
[1672470165] unbound[30234:0] info: query response was ANSWER
[1672470165] unbound[30234:0] info: prime trust anchor
[1672470165] unbound[30234:0] info: generate keytag query _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: resolving . DNSKEY IN
[1672470165] unbound[30234:0] info: resolving _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: response for . DNSKEY IN
[1672470165] unbound[30234:0] info: reply from <.> 202.12.27.33#53
[1672470165] unbound[30234:0] info: query response was ANSWER
[1672470165] unbound[30234:0] info: validate keys with anchor(DS): sec_status_bogus
[1672470165] unbound[30234:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672470165] unbound[30234:0] info: prime trust anchor
[1672470165] unbound[30234:0] info: generate keytag query _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: resolving . DNSKEY IN
[1672470165] unbound[30234:0] info: response for _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: reply from <.> 199.9.14.201#53
[1672470165] unbound[30234:0] info: query response was NXDOMAIN ANSWER
[1672470165] unbound[30234:0] info: response for . DNSKEY IN
[1672470165] unbound[30234:0] info: reply from <.> 199.7.91.13#53
[1672470165] unbound[30234:0] info: query response was ANSWER
[1672470165] unbound[30234:0] info: validate keys with anchor(DS): sec_status_bogus
[1672470165] unbound[30234:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672470165] unbound[30234:0] info: prime trust anchor
[1672470165] unbound[30234:0] info: generate keytag query _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: resolving . DNSKEY IN
[1672470165] unbound[30234:0] info: resolving _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: resolving _ta-4f66. A IN
[1672470165] unbound[30234:0] info: prime trust anchor
[1672470165] unbound[30234:0] info: response for . DNSKEY IN
[1672470165] unbound[30234:0] info: reply from <.> 199.9.14.201#53
[1672470165] unbound[30234:0] info: query response was ANSWER
[1672470165] unbound[30234:0] info: validate keys with anchor(DS): sec_status_bogus
[1672470165] unbound[30234:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672470165] unbound[30234:0] info: validate keys with anchor(DS): sec_status_bogus
[1672470165] unbound[30234:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672470165] unbound[30234:0] info: prime trust anchor
[1672470165] unbound[30234:0] info: generate keytag query _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: prime trust anchor
[1672470165] unbound[30234:0] info: resolving . DNSKEY IN
[1672470165] unbound[30234:0] info: response for _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: reply from <.> 192.36.148.17#53
[1672470165] unbound[30234:0] info: query response was NXDOMAIN ANSWER
[1672470165] unbound[30234:0] info: response for . DNSKEY IN
[1672470165] unbound[30234:0] info: reply from <.> 202.12.27.33#53
[1672470165] unbound[30234:0] info: query response was ANSWER
[1672470165] unbound[30234:0] info: validate keys with anchor(DS): sec_status_bogus
[1672470165] unbound[30234:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672470165] unbound[30234:0] info: validate keys with anchor(DS): sec_status_bogus
[1672470165] unbound[30234:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672470165] unbound[30234:0] info: prime trust anchor
[1672470165] unbound[30234:0] info: generate keytag query _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: resolving _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: prime trust anchor
[1672470165] unbound[30234:0] info: resolving . DNSKEY IN
[1672470165] unbound[30234:0] info: response for _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: reply from <.> 192.112.36.4#53
[1672470165] unbound[30234:0] info: query response was NXDOMAIN ANSWER
[1672470165] unbound[30234:0] info: response for . DNSKEY IN
[1672470165] unbound[30234:0] info: reply from <.> 198.41.0.4#53
[1672470165] unbound[30234:0] info: query response was ANSWER
[1672470165] unbound[30234:0] info: validate keys with anchor(DS): sec_status_bogus
[1672470165] unbound[30234:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672470165] unbound[30234:0] info: validate keys with anchor(DS): sec_status_bogus
[1672470165] unbound[30234:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672470165] unbound[30234:0] info: prime trust anchor
[1672470165] unbound[30234:0] info: generate keytag query _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: resolving _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: prime trust anchor
[1672470165] unbound[30234:0] info: resolving . DNSKEY IN
[1672470165] unbound[30234:0] info: response for _ta-4f66. NULL IN
[1672470165] unbound[30234:0] info: reply from <.> 198.97.190.53#53
[1672470165] unbound[30234:0] info: query response was NXDOMAIN ANSWER
[1672470165] unbound[30234:0] info: response for . DNSKEY IN
[1672470165] unbound[30234:0] info: reply from <.> 192.58.128.30#53
[1672470165] unbound[30234:0] info: query response was ANSWER
[1672470165] unbound[30234:0] info: validate keys with anchor(DS): sec_status_bogus
[1672470165] unbound[30234:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672470165] unbound[30234:0] info: validate keys with anchor(DS): sec_status_bogus
[1672470165] unbound[30234:0] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
[1672470165] unbound[30234:0] info: Could not establish a chain of trust to keys for . DNSKEY IN
Just watching this from the sidelines, thought I'd mention this is also a handy logging option:
val-log-level: <number> Have the validator print validation failures to the log. Regardless of the verbosity setting. Default is 0, off. At 1, for every user query that fails a line is printed to the logs. This way you can monitor what happens with validation. Use a diagnosis tool, such as dig or drill, to find out why validation is failing for these queries. At 2, not only the query that failed is printed but also the reason why unbound thought it was wrong and which server sent the faulty data.
192.5.5.241 is indeed f.root-servers.net, the others are root server, too, so this is fine, however,
is really bad.
It makes me suspicious that anything in your house (router, firewall) is intercepting DNS queries and reroutes them to somewhere else. Can this be the case? If not, your ISP might be doing this for you.
Please run the following commands on your Pi-hole: