Add CACHE_SIZE to setupVars.conf

There’s enough cache in Unbound that I’ve never seen it fill. About 75% of Unbound’s replies come from cache in my setup probably because I have it set to serve expired.

I’ve found that the TTL on most DNS records are set low enough that Pi-Hole has to ask Unbound for an answer frequently enough that it still learns what is popular.

An answer from Pi-Hole cache is always slightly faster than one from Unbound cache in my informal testing.

That's correct, there's an additional application to go through, but Unbound can prefetch queries before they expire and thereby always serving up-to-date (ish) records. If you enable caching in dnsmasq then only a limited number of requests is sent to it and it doesn't really know what is popular and what isn't. I don't have a lot of RAM for pihole + unbound and it's running in a busy network so records need to be purged from cache regularly.

@sawsanders Yes, an answer from pihole cache is indeed faster but it's not much of an improvement. With pihole caching enabled an answer from cache is returned in 1.9ms, with Unbound as cache it's 1.95ms. This 0.05ms is easily worth the tradeoff of having a more reliable prefetch for me. Tested using namebench.

I'll see if I can get something going to add this.

I don't know if this is true, but I read several times that the prefetch in Unbound only works if the requests come from multiple and different clients - which is not the case with Pi-hole (= 1 client).

In my home environment Unbounds cache hit ratio with prefetch: yes & serve-expired: no is extreme low (<10%).

It works fine for me with one client (pihole), cache hit ratio is usually around 40% here. Nowhere in the Unbound documentation says it works any differently either. Even after a manual unbound restart it quickly picks up and prefetches stuff.

total.num.cachehits=60
total.num.cachemiss=59
total.num.prefetch=50

Are those values in percentages or actual queries/records. 60% cache hit is significant, 60 actual cache record hits is statistically insignificant.

Those numbers are actual queries. It's showing 50% cache hit.

Here's my Unbound statistics from 6 days of uptime:

total.num.queries=113770
total.num.cachehits=100633
total.num.cachemiss=13137
total.num.prefetch=83041
total.num.recursivereplies=13137

Indeed, it were just the total numbers immediately after a restart (< 1m uptime). After running for 2 hours (still nothing compared to previous stats):

total.num.queries=2553
total.num.cachehits=2095
total.num.cachemiss=458
total.num.prefetch=1413

Caching + prefetching is working well (as it did before :slight_smile:). Since I've added Unbound the dns response times have been greatly reduced. Of course that's mostly for repeatedly visited websites, but that's what matters to me as I don't really browse that many new pages on a daily basis.

I know the Devs don't recommend disabling the cache - I would like to know why you don't recommend it. Is there a specific reason except for answers might a bit faster coming from pihole's cache than unbound's cache?

You assume everyone uses Unbound.

To expand. A properly functioning cache will store for the life of the TTL value. Doesn't matter who caches it. If Pi-hole is directly querying Unbound then both Pi-hole and Unbound will have the same data for the same amount of time.

Ok, I wasn't specific enough - as you pointed out: why don't you recommend disabling the cache in conjunction with Unbound? (I assumed this limitation was obvious as this whole topic was about pihole together with unbound.)

So you still wouldn't recommend it - but it also wouldn't harm any function of pihole?

What I don't get then is the recommended setting for Unbound prefetch: yes.
If both caches contain the same data for the same amount of time queries will be answered by pihole's cache until TTL expires and only afterwards forwarded to Unbound where TTL has also expired. But prefetch: yes will only work if Unbound gets queried for a cached hostname which is going to expire in less than 10% of its TTL. But this would never happen as with >10% TTL the query will be answered by pihole. However I see Unbound does prefetch domains (with both caches enabled) - but where do the necessary queries come from? Does pihole generate those requests as part of a prefetch algorithm?

No. Pi-hole does not perform any pre-fetching. It answers DNS queries as they arrive. If the answer is in cache, the cached answer is served. If not, then Pi-hole goes to the upstream resolver and gets the answer.

Cost / benefit ratio.

3 people have voted for this. The potential savings is on the order of fractions of a nanosecond. Not worth it to me.

This is what I expected - and that makes it so confusing for me. Still don't know how prefetching should work if another cache answers all queries within TTL.

I recommend inquiring with the unbound folks.

Pi-hole still queries Unbound for the initial query. Unbound has that in it's cache. Unbound will prefetch what it has in its cache.

Quoting the unbound.conf man page:

prefetch: <yes or no>
If yes, message cache elements are prefetched before they expire
to keep the cache up to date. Default is no. Turning it on
gives about 10 percent more traffic and load on the machine, but
popular items do not expire from the cache.

There is no explanation of "popular items". This is always the time where you'll have to look into the source code (so I did this). It's not simple to say this as the situation between "ordinary" DNS and "mesh" DNS is somewhat different, however, at first glance, it seems that all DNS queries are prefetched when they are close to TTL expiration. One could consult total.num.prefetch and compare it to total.num.queries to get some experimental data (unbound-control).

edit See also: [Unbound-users] About the prefetch switch used in the unbound.conf.
This explanation makes it clear that prefetching might not have an effect with caching enabled. It may be a valid point to argue for disabling Pi-hole's cache. However, mind that DNSSEC will not be available without a cache as the cache is integral to DNSSEC validation. The validation requires the use of a whole set of RRsets. The only way to have all those RRsets is to put them in the cache or you'd generate a ton of requests each time. This would surely be disadvantageous.

1 Like
dschaper@Mariner-10:~$ dig +dnssec sigok.verteiltesysteme.net @192.168.88.2

; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> +dnssec sigok.verteiltesysteme.net @192.168.88.2
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62379
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1472
;; QUESTION SECTION:
;sigok.verteiltesysteme.net.    IN      A

;; ANSWER SECTION:
sigok.verteiltesysteme.net. 60  IN      A       134.91.78.139
sigok.verteiltesysteme.net. 60  IN      RRSIG   A 5 3 60 20200630020001 20200331020001 30665 verteiltesysteme.net. PwtSs7j2fILKCP9RAvfsCHmaZRHDsILfRNSwaoiGtIWJPMK+Jnlf0/lf CkvkYqPwcSV4kO7HYd4Sn/FimHw7C8T5uxw8BglafdemlfgapK2q0mvw ZiNcviBTptQvYTqPlIEhCvzgbicVACc8NadF1QDEHprqjfTqxoZMe/Go Vew=

;; Query time: 403 msec
;; SERVER: 192.168.88.2#53(192.168.88.2)
;; WHEN: Tue Apr 14 15:07:46 PDT 2020
;; MSG SIZE  rcvd: 251

dschaper@Mariner-10:~$ dig +dnssec sigok.verteiltesysteme.net @192.168.88.2

; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> +dnssec sigok.verteiltesysteme.net @192.168.88.2
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46776
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1472
;; QUESTION SECTION:
;sigok.verteiltesysteme.net.    IN      A

;; ANSWER SECTION:
sigok.verteiltesysteme.net. 55  IN      A       134.91.78.139
sigok.verteiltesysteme.net. 55  IN      RRSIG   A 5 3 60 20200630020001 20200331020001 30665 verteiltesysteme.net. PwtSs7j2fILKCP9RAvfsCHmaZRHDsILfRNSwaoiGtIWJPMK+Jnlf0/lf CkvkYqPwcSV4kO7HYd4Sn/FimHw7C8T5uxw8BglafdemlfgapK2q0mvw ZiNcviBTptQvYTqPlIEhCvzgbicVACc8NadF1QDEHprqjfTqxoZMe/Go Vew=

;; Query time: 7 msec
;; SERVER: 192.168.88.2#53(192.168.88.2)
;; WHEN: Tue Apr 14 15:07:51 PDT 2020
;; MSG SIZE  rcvd: 251

dschaper@Mariner-10:~$ dig chaos txt version.bind @192.168.88.2

; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> chaos txt version.bind @192.168.88.2
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1183
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;version.bind.                  CH      TXT

;; ANSWER SECTION:
version.bind.           2       CH      TXT     "dnsmasq-pi-hole-2.81"

;; Query time: 2 msec
;; SERVER: 192.168.88.2#53(192.168.88.2)
;; WHEN: Tue Apr 14 15:09:56 PDT 2020
;; MSG SIZE  rcvd: 74

Screenshot_2020-04-14 Pi-hole Admin Console(2) Screenshot_2020-04-14 Pi-hole Admin Console(1) Screenshot_2020-04-14 Pi-hole Admin Console

Thank you for your effort.

In my opinion your observation what's in the source code does not match the explanation given in the link (I believe the explanation is wrong/outdated, not the code :slight_smile:).

  1. The source code suggests that all DNS queries are prefetched. This in line with what I see because a lot gets prefetched.
total.num.queries=5292
total.num.cachehits=4460
total.num.cachemiss=832
total.num.prefetch=2735

--> Disabling Pihole's cache is not necessary, prefetching happens anyway.

  1. The deeper explanation given in the link is in line with unbound.conf man where 10% more traffic is expected as

If a client requests the dns lookup and the TTL of the cached hostname is going to expire in less than 10% of its TTL, unbound will (1st) return the ip of the host to the client and (2nd) pre-fetch the dns request from the remote dns server. This method has been shown to increase the amount of cached hits by local clients by 10% on average."

--> Disabling Pihole's cache might be beneficial as Unbound would be queried already for "close-to expiring" entries and not only for "already expired" entries

As I actually see 1) it leads me to the conclusion that the explanation given in 2) is wrong/outdated (they might have changed the behavior at some point).

Conclusion: Prefetching works also with Pihole's cache enabled and aside from a theoretical tiny tiny speed improvement there are no additional benefits from disabling it (but negative side effects on DNSSEC might occur)