pihole-FTL + unbound with Cache DB Module Options (Redis)

Hmmmmm, I don’t have the impression that Redis keeps track of TTL. It is unbound seeing that the TTL is expired.

See my earlier remarks about serving expired info by Unbound and Knot. So it emulates Redis and in the mean time it gets fresh info for the next request.

serve-expired: <yes or no>
              If enabled, unbound attempts to serve old responses  from  cache
              with  a  TTL of 0 in the response without waiting for the actual
              resolution to finish.  The actual resolution answer ends  up  in
              the cache later on.  Default is "no".
serve-expired-ttl: <seconds>
              Limit  serving  of expired responses to configured seconds after
              expiration. 0 disables the limit. This option only applies  when
              serve-expired is enabled. The default is 0.
serve-expired-ttl-reset: <yes or no>
              Set  the  TTL  of expired records to the serve-expired-ttl value
              after a failed attempt to retrieve  the  record  from  upstream.
              This  makes sure that the expired records will be served as long
              as there are queries for it. Default is "no".

Found a discussion here which might explain some things.

@GieltjE: On tweakers.net, you stated (translation) as soon as de Redis cache provides an expired record, a new query is performed. This appears to be correct, but the request is (see below) performed after the response is provided to the client, this to update the expired entry (see below - prefetch=yes).

I, and probably others, added prefetch=yes to my unbound config, as the result of a discussion (@DL6ER - Everyone of the team agreed to remove the lines) , where it was decided to remove cache-min-ttl and cache-max-ttl from the recommended unbound config, so the behavior @GieltjE has seen is probably because of the prefetch=yes setting.

The expired record will be served to the client only, if the setting serve-expired: yes is configured, as pointed out by @msatter.

So I changed my unbound config, ensuring the following settings are already there or added, and there are no cache-xxx-ttl settings in the config.:

prefetch: yes
serve-expired: yes
serve-expired-ttl: 0
serve-expired-ttl-reset: yes

Some questions remain, regarding the use of Redis, as pointed out by @DL6ER (I still don’t see the need to add more moving gears)

Restarted unbound, emptied the Redis cache (redis-cli FLUSHALL), so it’s to soon to tell what the result will be, however, it remains fascinating and worth discussing…

So far I used

to limit serving, not serving too old records and force a refresh after some time.

Redis users, be aware of the fact that Redis writes a lot of log entries (when performing database saves) to /var/log/redis/redis-server.log

to over come this (killing your SD card) I added the following

/etc/fstab

tmpfs /var/log/redis tmpfs nodev,nosuid,gid=adm,uid=redis,mode=2750,size=16M 0 0

/etc/logrotate.d/redis-server

/var/log/redis/redis-server*.log {
        daily
        missingok
        rotate 1
        compress
        notifempty
}

continuing to try to understand Redis

first, increase unbounds verbosity: sudo /usr/sbin/unbound-control verbosity 5

Don’t forget to decrease the verbosity or restart unbound, after you’re done, or your unbound log will explode.

some entries found in the unbound log:

Oct 21 12:29:39 unbound[15130:0] debug: redis_lookup of BB4FC6A97D927FBBAC80E218                                                                                                             B91B9A126683A2B795378885CEF766EC85E81FDE
Oct 21 12:29:39 unbound[15130:0] debug: redis_lookup: no data cached

Oct 21 12:29:39 unbound[15130:0] debug: redis_store DC9B09CE18ABA23DB4FC71D86033A74F67B73D9AA106BECA6192C0512AE16079 (73 bytes)
Oct 21 12:29:39 unbound[15130:0] debug: redis_store set completed

Oct 21 12:34:23 unbound[15130:0] debug: redis_lookup of 8D825D767F185F94C89F48EC5C5A87BC3751CA17DEDE6940ABF62EEF2A5C209D
Oct 21 12:34:23 unbound[15130:0] debug: redis_lookup found 875 bytes
Oct 21 12:34:23 unbound[15130:0] info: redis ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0

These are random entries (not related) from the log. Especially the last one proves Redis is really doing something, even on a system where unbound hasn’t been up very long (see my last entry, regarding prefetch=yes)

Oh yes, but you see there also the reason for why it still wasn’t changed - I put it on some paper and then forgot. Problem is that I’m traveling as well now. If you want to submit a docs PR I’d be happy approving and merging it.

Is this a also starting from an empty Redis database?

I’m repeating myself here, but I still don’t see what Redis effectively brings into play with serve-expired: yes besides a possible restarting resistance.

For me that is already a reason or advantage.

I noticed another, probably important sentence in the unbound documentation:

The thread waiting for a response from the Redis server cannot handle other DNS queries.

To my understanding, this means a change to the num-threads setting will also be required, this to avoid delays

num-threads: 2

anybody agrees / disagrees?

Where can I find instructions on how to do that without causing problems?

To ensure I don’t need to troubleshoot Redis problems, I’ve added the following:

  1. Script, /etc/unbound/clearRedisDB.sh, content:

#!/bin/bash
redis-cli FLUSHALL

  1. Add line to /lib/systemd/system/unbound.service, content:

ExecStartPre=/etc/unbound/clearRedisDB.sh

This empties the Redis cache whenever unbound starts, so both caches where empty before this log entries where entered.

Indeed “prefetch: yes” is in my unbound config, but the serve-expired rows are not present.
What is the best config to adhere to the RFC’s?

Answering my own question here. Found this document on how to optimize unbound
On a raspberry pi 3B and 3B+, this would imply the correct setting on a 3B would be num-threads: 4

With the great help of @mibere (hank you for this), I’ve made some changes to my unbound config. This implies unbound needs to be compiled with an additional option: --with-libevent, which requires the installation of libevent-dev (sudo apt-get -y install libevent-dev).

I’ve already applied some of the proposed settings, not all (yet), and have updated my GitHub repository to reflect the changes.

Feel free to comment on / discuss the settings in these configuration files.

Not sure, but I don’t expect Redis to have a great impact (if any), when these settings are not present. To verify this, you could increase the verbosity of unbound, and check the unbound log for entries containing redis_lookup found (sudo cat /etc/unbound/unbound.log | grep redis - use the correct location). It’s a lot of logging (verbosity 5), so ensure you have sufficient disk space…
I’m still using my initial settings, evaluating this and keeping the proposed settings from @mibere in mind (see above).

Fork https://github.com/pi-hole/docs and submit a PR against master (there is no development branch on this repo). After the PR has been merged, the changes will be automatically deployed to https://docs.pi-hole.net

Thanks for this information. It is useful