Unbound seems to only be partially working

[EDIT: This thread was split into two different discussions and the first paragraph was the "solution" to the original topic. The second paragraph is the subject of this particular thread]

So good news/bad news:

Both of those extra lines you and chrislph identified were from setup tutorials that seem to have steered me wrong (I'd originally used them because DNSSEC tests on my first try setting up unbound were mixed/failing and I thought the original tutorial I'd used may have been missing necessary steps). Removing the additional "include" line didn't fix unbound, so I eventually decided to start from scratch and do a complete reinstall of my RP4 (including reinstalling pihole + a teleporter settings restore) and strictly followed the official unbound tutorial steps. That's the good news: the reinstall is done and unbound seems to be working.

Here's the bad news: I'm still getting extremely mixed results on DNSSEC testing! Dig commands return NOERROR and https://www.dnssec.cz/ and https://dnssec-tools.org/test/ are both successful, but DNSSEC Resolver Test and dnscheck.tools - check your dns resolvers both show failures and What Is My DNS Server? Check Your DNS Server Address doesn't return my own public ip as my DNS server.

So I'm baffled: Unbound seems to only be partially working?

You know the drill :wink:

unbound-checkconf

sudo rgrep -v '^ *#\|^$' /etc/unbound/unbound.conf*

?

Also is time/date acurate?

timedatectl

Might also want to check time/date on the PC that you're using for testing below links:

I have no clue what those tests do exactly.
But do below two answer similar with SERVFAIL for the first and NOERROR + IP 5.45.107.88 for the second?

$ dig +noall +comments fail01.dnssec.works @127.0.0.1 -p 5335
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 24382
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
$ dig +noall +comments +answer dnssec.works @127.0.0.1 -p 5335
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19729
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; ANSWER SECTION:
dnssec.works.           3600    IN      A       5.45.107.88

If so, I wouldnt be too concerned about results for those tests.

What failures do you see with that one? What DNS resolvers does it show? It should just be your public IP address. When I run that one with Unbound I just have my public IP and all 12 green ticks are shown.

What address does it return?

Another one you can try is DNS leak test. Run the Extended test. If you are only using Unbound then you will see your own public IP as the server (and likely your ISP's name alongside it). But if you are using other DNS servers as well, without knowing, this test tends to show them up.

Note that that reference to "leaks" is in relation to VPNs, so using the tool this way isn't testing for leaks, but it's still a useful tool to try.

If you have more than one computer using your network, and using Pi-hole for DNS, then by all means try the tests on there too. They should be giving consistent results.

From what you've described I suspect you have another DNS server still in use without realising it. Double check your settings in Settings > DNS > Upstream, and check you didn't manually change any client DNS settings while you were testing the original Unbound problem and have accidentally left them in.

If you want to create a debug log it might highlight something. It's pihole -d or Tools > Generate debug log and then post just the token URL here.

unbound-checkconf shows no errors :slightly_smiling_face:

timedatectl is correct

sudo rgrep -v '^ *#\|^$' /etc/unbound/unbound.conf*
WaynePi2@pihole2:~ $ sudo rgrep -v '^ *#\|^$' /etc/unbound/unbound.conf*
/etc/unbound/unbound.conf:include-toplevel: "/etc/unbound/unbound.conf.d/*.conf"
/etc/unbound/unbound.conf.d/pi-hole.conf:server:
/etc/unbound/unbound.conf.d/pi-hole.conf:    verbosity: 0
/etc/unbound/unbound.conf.d/pi-hole.conf:    interface: 127.0.0.1
/etc/unbound/unbound.conf.d/pi-hole.conf:    port: 5335
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-ip4: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-udp: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-tcp: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    do-ip6: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    prefer-ip6: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    harden-glue: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    harden-dnssec-stripped: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    use-caps-for-id: no
/etc/unbound/unbound.conf.d/pi-hole.conf:    edns-buffer-size: 1232
/etc/unbound/unbound.conf.d/pi-hole.conf:    prefetch: yes
/etc/unbound/unbound.conf.d/pi-hole.conf:    num-threads: 1
/etc/unbound/unbound.conf.d/pi-hole.conf:    so-rcvbuf: 1m
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 192.168.0.0/16
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 169.254.0.0/16
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 172.16.0.0/12
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: 10.0.0.0/8
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: fd00::/8
/etc/unbound/unbound.conf.d/pi-hole.conf:    private-address: fe80::/10
/etc/unbound/unbound.conf.d/root-auto-trust-anchor-file.conf:server:
/etc/unbound/unbound.conf.d/root-auto-trust-anchor-file.conf:    auto-trust-anchor-file: "/var/lib/unbound/root.key"

Each of those "dig" tests return the expected results (SERVFAIL and NOERROR respectively)

1 Like

dnscheck.tools output on a Windows client:


^This is not my public IP.
--A separate Windows client shows similar results with a different IP address attributed to Leaseweb
--Chrome on a wifi-connected Android shows a dozen different DNS resolvers attributed to "WoodyNet" and 12 green checks)

What is my DNS Server?
Shows similar results: a different Leaseweb address for each Windows client and Chrome on Android errors out

DNS leak test - Extended
More of the same.

Upstream DNS
127.0.0.1#5335 [Unbound] in the Custom 1 (IPv4) field is the only checked Upstream DNS on the whole page.

debug log
https://tricorder.pi-hole.net/H1WzfMBT/

Thanks for running those tests and the debug log. Out of interest do you recognise Leaseweb? Are they an organisation you use?

Your debug log shows that Pi-hole is working correctly and blocking domains when it is used as the DNS server. The sections of your pihole.log show that some devices on the network are using Pi-hole, with some domains being blocked and some being forwarded to Unbound. All looks good.

Your Pi-hole and Unbound are on 10.0.0.3 and your router is on 10.0.0.1. Your router is the DHCP server and looks like it has been configured with a reserved address for the Pi-hole (to make sure Pi-hole always gets .3). The router's DHCP is correctly giving out the Pi-hole's .3 address to clients as the DNS server to use.

However it is also giving out the address 10.0.0.2. This means clients will use both your Pi-hole and whatever that machine is for their DNS.

      dns-server: 10.0.0.2
      dns-server: 10.0.0.3

This looks a likely culprit for what you are seeing. Can you identify what that machine is at .2? Is it indeed running a DNS server? Pi-hole's network view in Tools > Network might be helpful here, it will show you what it can see and can help identify the vendor.

On the Windows client what are the outputs of the following commands? You can use the mouse to select and copy the text from the terminal and paste it into your reply.

nslookup pi.hole
nslookup flurry.com 10.0.0.3
nslookup flurry.com 10.0.0.2

Nope, I've never heard of Leaseweb before. My ISP is Verizon Fios and I'm located in central PA.

.2 and .3 are both RP4B+ devices configured identically with PiHole, Unbound, and Gravity Sync: they were configured using the exact same steps within 48 hours of each other (this thread originally started when I was running into problems with .3, which is the one I used to figure everything out before following the same process on .2).

I've tried repeating all of the above tests (and others) in three combinations:

  • .2 online, .3 powered off
  • .3 online, .2 powered off
  • both .2 and .3 online (which was the case at the moment those logs were recorded)

The results are identical in all cases, which is what we'd expect with both RP4s configured identically.

Looks good.

Thanks, okay that's good news and explains the .2.

On the Windows system where you performed the tests, what DNS servers are showing when you run the command below.

ipconfig /all

Here's an example of what you're looking for.

It should just show one or both of your Pi-holes. Are there any other DNS servers listed?

The two PiHoles are the only two DNS servers listed in ipconfig on both of the Windows clients and the equivalent menu on the wifi Android device.

I'm at a loss as to why you are seeing unknown DNS servers when you have clients (or at least this one client) using only your working Pi-holes and working recursive Unbounds. When testing from that client you should be seeing your own public IP in these tests, because Unbound is performing recursive lookups from your network and that's how your network looks to those test sites.

But instead you are seeing other unknown servers and, as might be expected from unknown servers, varied results in terms of DNSSEC support. It makes me think that your Windows clients have more going on in terms of how their DNS is working.

Do you have any Linux clients using the Pi-hole for DNS? They are easier to test with than Windows clients.

When you go to a website on your Windows client – say, example.com – are you seeing that domain appear in your Pi-hole Query Log when you refresh the page?

What is the result of this command when run from your Windows client?

nslookup -class=chaos -type=txt version.FTL

It should return an IP address for one of your Pi-holes and give the FTL version. Eg mine replies (layout can vary slightly):

Server:		192.168.1.2
Address:	192.168.1.2#53
version.FTL	text = "v5.23"

What about this command?

nslookup -class=chaos -type=txt version.bind 198.41.0.4

It should return

Server:		198.41.0.4
Address:	198.41.0.4#53
version.bind	text = "ATLAS"

And finally

nslookup -class=chaos -type=txt version.bind 192.33.4.12

should return

Server:		192.33.4.12
Address:	192.33.4.12#53
version.bind	text = "c-root"

If that checks out then next step is going to be more verbose Unbound logging and some tests.

1 Like

Sadly no Linux clients available at the moment.

Those other tests seem to have returned some interesting results though, meaning none of them gave the expected results:

I am not finding Example.com show up in the query logs of either pihole when I tested it just a moment ago.

nslookup -class=chaos -type=txt version.FTL
C:\Users\Shawn>nslookup -class=chaos -type=txt version.FTL
Server:  pi.hole
Address:  10.0.0.2

*** pi.hole can't find version.FTL: Query refused

Note: Both PiHole webadmins are reporting an Active status with no issues reported and FTL v5.23

nslookup -class=chaos -type=txt version.bind 198.41.0.4
C:\Users\Shawn>nslookup -class=chaos -type=txt version.bind 198.41.0.4
Server:  a.root-servers.net
Address:  198.41.0.4

Non-authoritative answer:
version.bind    text =

        "unbound 1.13.2"

[Note: please excuse the forum software formatting the 'unbound 1.13.2' differently above and in the snippet below: it seems the quotation marks which are present in the cmd output are throwing it off]

nslookup -class=chaos -type=txt version.bind 192.33.4.12
C:\Users\Shawn>nslookup -class=chaos -type=txt version.bind 192.33.4.12
Server:  c.root-servers.net
Address:  192.33.4.12

Non-authoritative answer:
version.bind    text =

        "unbound 1.13.2"

Finally a clue, I hope?

Yes its a mess :wink:

C:\>nslookup -class=chaos -type=txt version.FTL
Server:  pi.hole
Address:  10.0.0.2

version.FTL     text =

        "v5.22"
C:\>nslookup -class=chaos -type=txt version.bind 198.41.0.4
[..]
version.bind    text =

        "ATLAS"
C:\>nslookup -class=chaos -type=txt version.bind 192.33.4.12
[..]
version.bind    text =

        "c-root"

The top one not replying with the proper answer might indicate some DNS filtering going on.
Check your router settings for security related DNS settings!
Or DNS redirect/filter settings (it has many names).
If DNS queries from your clients are redirected first, it might explain why the DNSSEC test links are not working.

EDIT: Also an antivirus or browser security/ads-blocker plugin on that client thats used for testing could interfere (though less likely in this case).

Select/highlight the code output here on Discource after pasting and use the </> button to format the output before posting.

EDIT: Fixed it for you :wink:

I didt know you could do that :+1:

Its missing vital info though :wink:

pi@ph5b:~ $ dig +short @localhost chaos txt authors.FTL
pi@ph5b:~ $
pi@ph5b:~ $ dig +short @localhost chaos txt authors.bind
"Simon Kelley"

Ow does below one show the exact same "unbound 1.13.2" version when run on that MS client?

nslookup -class=chaos -type=txt version.bind

And is the answer "dnsmasq-pi-hole-XXX" displayed if run below on the Pi-hole host?

dig +short @localhost chaos txt version.bind

EDIT: Ow one more:

dig +short @$(hostname -I) chaos txt version.bind

If they differ I'm pretty sure something is filtering DNS.

You've confirmed the ipconfig output show that the Pi-holes at .2 and .3 are the only configured DNS servers. All this implies that something running on your Windows client is intercepting DNS requests and sending them directly to the Leaseweb DNS servers. The requests never get to your Pi-hole or Unbound.

Another possibility might have been that your router was doing a similar kind of redirection, or even that your ISP was interfering with the traffic, but in that case the requests would have first gone to Pi-hole and been sent upstream to your Unbound, and then interfered with after that, so you would still be seeing your tests in your Pi-hole logs.

I welcome any thoughts from the devs/mods in case I've overlooked something in all this.

Are you running any VPN software, anything like that? Any DNS or "helper" tools, any privacy tools, anything like that? Sometimes these kind of apps take it upon themselves to reconfigure how things work. Do you have antivirus and can you scan your computer? Don't be alarmed by asking that, it's worth checking though because redirecting DNS to a hosted instance is also something that malware might do.

Your nslookups...

This chaos class txt query for version.FTL is something only Pi-hole understands. Your result shows that nslookup is asking one of your Pi-holes (in this case the one at .2) which is correct since that is one of the configured DNS servers. But this request gets redirected and the server that answers is not Pi-hole and so cannot make sense of this query and returns a refused response. If the Pi-hole had answered it would have returned its FTL version.

Pi-hole:

$ nslookup -class=chaos -type=txt version.FTL 192.168.1.2
Server: 192.168.1.2
Address: 192.168.1.2#53
version.FTL text = "v5.23"

Random external server:

$ nslookup -class=chaos -type=txt version.FTL 9.9.9.9
Server: 9.9.9.9
Address: 9.9.9.9#53
** server can't find version.FTL: REFUSED

The other two nslookup should give the strings ATLAS and c-root but instead they returned an Unbound version, implying that again a different unexpected server replied. Unbound is commonly used by providers.

As deHakkelaar suggest try running one of these version nslookup without specifying a server. That would normally cause your Pi-hole to reply with its version, which should look something like this:

$ nslookup -class=chaos -type=txt version.bind
Server: 192.168.1.2
Address: 192.168.1.2#53
version.bind   text = "dnsmasq-pi-hole-v2.89-9461807"   <-- this has come from Pi-hole

but if the query is being directed away from Pi-hole to another server, you'll just get that previous response instead, ie

text = "unbound 1.13.2"

Chris is correct:
Your observation indicates that clients are by-passing Pi-hole.

This often happens if your browser enables DNS-over-HTTPS.
But in your case, as your nslookup results do not return the expected results, even when you explicitly request resolution via DNS root servers, that would mean that this is done by redirecting DNS requests.

Forceful redirection of DNS requests can be done by software on your client, your router or your ISP (e.g. if you subscribed to ISP-side parental control services), and it would also be common for VPN service providers (in an effort to prevent DNS leakages).

You probably should check all of those potential sources, but start with:

Indeed, the last time I saw WoodyNet appearing in a forum report here, that was due to an antivirus software package enabling a feature named Avast RealSite.

If you are runnng some antivirus, you should disable its DNS service features when running with Pi-hole.

2 Likes

Still not a full fix, but finally a step in the right direction!

I started disabling any AVG-related "features" I could find that might have any effect on DNS and none of them made any difference until I found "Fake Website Shield" buried deep within the "Full protection" settings menu. Disabling this function immediately changed my dnsleaktest.com and dnscheck.tools results from a single Leaseweb server as you saw above to now resemble the results from my Android device: a handful of WoodyNet servers with all green check boxes:


And the updated cmd tools from deHakkelaar:

C:\Users\Shawn>nslookup -class=chaos -type=txt version.FTL
Server:  pi.hole
Address:  10.0.0.2

version.FTL     text =

        "v5.23"

C:\Users\Shawn>nslookup -class=chaos -type=txt version.bind 198.41.0.4
Server:  a.root-servers.net
Address:  198.41.0.4

version.bind    text =

        "dnsmasq-2.86"

C:\Users\Shawn>nslookup -class=chaos -type=txt version.bind 192.33.4.12
Server:  c.root-servers.net
Address:  192.33.4.12

version.bind    text =

        "dnsmasq-2.86"

example.com is also now showing in my pihole logs if I try accessing it from an incognito window and/or browser which didn't already have it in the cache (such as the ones I used for testing yesterday).

That said, we're still not quite there considering I'm still not seeing my own public IP in the DNS results and those latter two nslookups still aren't returning the expected results, but I wanted to report back on this first incremental step for the sake of anyone else who might run into this issue. Next up is to do a much deeper dive into my routers' settings (temporarily running my new Ubiquiti UDM-SE double-NAT'd behind a Verizon Fios ActionTec while I learn/configure the much more complicated Ubiquiti. Both PiHoles are connected to the Ubiquiti, as are the clients I'm using for testing).

Update 2: a solution!

Aside from the AVG setting identified above, the other culprit turned out to be in the Ubiquiti "Firewall & Security" menu: a setting called "Ad blocking" that redirects DNS to the WAN DNS setting (seems to default to Cloudflare if in Auto mode)!

Turning this off immediately started returning my own public IP address in dnsleak.com regardless of the Verizon router's WAN/DHCP settings or the Ubiquiti's WAN setting (obviously DHCP DNS is set to the PiHole addresses)

Also, for anyone else who stumbles across this, after some googling I learned that WoodyNet is a part of the Quad9 network.

My settings going forward, for anyone with similar hardware:
Verizon Actiontec WAN DNS: 9.9.9.9
Verizon Actiontec DHCP DNS: 9.9.9.9 (since PiHoles aren't accessible outside of the Ubiquiti LAN: this is only temporary anyway until I make the Ubiquiti the primary router)

Ubiquiti WAN DNS: 9.9.9.9
Ubiquiti DHCP DNS: PiHole addresses .2 and .3
Ubiquiti Firewall "ad blocking": DISABLED (other firewall settings seem to be fine and are enabled)

AVG Fake Website Shield (buried in Full Protection settings menu): DISABLED.

Thanks for all the help!!!!!!

Nice one, good detective work! Here's a summary of what you should be seeing when testing from the PC with everything fixed.

  • dnsleaktest and the other sites only show your own public IP as the DNS server (because that's what Unbound running on your own LAN looks like to them)
  • Each test and lookup you do appears in your Pi-hole Query Log
  • The various nslookup tests all return the correct results

In particular do these two tests now return "ATLAS" and "c-root"?

nslookup -class=chaos -type=txt version.bind 198.41.0.4
nslookup -class=chaos -type=txt version.bind 192.33.4.12

For anyone reading in future, these are just two of the current root name servers ("a" and "c") and these commands query them directly. If nothing is intercepting DNS then they will be the servers that respond. "a" will respond with "ATLAS" and "c" will respond with "c-root". But if something is intercepting DNS then the intercepting server will respond instead, usually with some kind of version string as we saw above. It's a useful way to help determine if something is hijacking DNS or if an ISP is intercepting it.

As for the software doing this, while it might be well-intentioned it's concerning to find out that it is vacuuming up all DNS requests and diverting them to its own services. It's quite a privacy concern and without having installed something like Pi-hole, no-one would ever know it was happening.

Here's the info on what AVG's Fake Website Shield is doing and how it works. It's enabled by default, so you would not know it was even there, and works by redirecting your DNS to AVG's own server "to prevent hijacking"!