As always, please read through the changelog before updating with pihole -up.
Highlights
More details for your adlists
The web dashboard does now provide health-status and statistics about downloaded and processed adlists. You can see when they were last downloaded, when they were last changed and it they work at all or contain invalid domains. This aims at simplifying using additional lists.
This PR adds a third default theme, an even `darker` variant of the dark one. It also fixes an issues with the existing `dark` theme where links in box headers (like the Query Log’s “show all”) were completely invisible.
Comparison between the existing dark (on the left) and the new darker (on the right):
Until now, FTL’s IP and IP-NODATA-AAAA blocking modes sourced the IP to deliver on a blocked domain from the setupVars.conf values IPV4_ADDRESS and IPV6_ADDRESS. This is, however, quite a limitation, especially if the device running Pi-hole has more than one interface or of the address is changing. To address this, FTL v5.8 implements an automated IP blocking mode. Instead of reading the addresses from setupVars.conf, we dynamically determine the address of the interface a query arrived on. We then use this IP address in the blocked reply. This does not only reduce maintenance (IPV4_ADDRESS and IPV6_ADDRESS can now be removed from setupVars.conf) but also localizes blocked queries. The automated detection can be overwritten using REPLY_ADDR4 and REPLY_ADDR6 in pihole-FTL.conf.
Update embedded dnsmasq to v2.85
New features, bug fixes and several other tweaks are contained in this release. Check out the change log below for more information.
Various enhancements and a few memory-leak fixes #1084
Resize shared memory only when locking #1072
This is not really a functional change, however, it makes the code more read- and understandable in some places.
Ensure FTL can be compiled from static tarballs #1091
Use properly-sized buffer for format_time() #1088 (thanks @bershanskiy)
Fix pihole-FTL test not terminating properly (noticed in a docker environment) #1067
Fix incorrect “FATAL: Trying to access upstream ID -1” warning in the logs #1061
DNS server improvements:
Fix problem with DNS retries in earlier versions.
The new logic in 2.83/2.84 which merges distinct requests for the same domain causes problems with clients which do retries as distinct requests (differing IDs and/or source ports.) The retries just get piggy-backed on the first, failed, request.
The logic is now changed so that distinct requests for repeated queries still get merged into a single ID/source port, but they now always trigger a re-try upstream.
To avoid excessive spamming of the upstream server, similar queries are collected for up to two seconds when the query has already been sent upstream. All clients will get replies when the answer comes back.
Avoid treating a dhcp-host which has an IPv6 address as eligible for use with DHCPv4 on the grounds that it has no address, and vice-versa.
Add dynamic-host option A and AAAA records which take their network part from the network of a local interface. Useful for routers with dynamically prefixes.
Teach bogus-nxdomain and ignore-address to take an IPv4 subnet.
Use random source ports where possible if source addresses/interfaces in use. CVE-2021-3448 applies.
It’s possible to specify the source address or interface to be used when contacting upstream name servers: server=8.8.8.8@1.2.3.4 or server=8.8.8.8@1.2.3.4#66 or server=8.8.8.8@eth0, and all of these have, until now, used a single socket, bound to a fixed port. This was originally done to allow an error (non-existent interface, or non-local address) to be detected at start-up. This means that any upstream servers specified in such a way don’t use random source ports, and are more susceptible to cache-poisoning attacks.
We now use random ports where possible, even when the source is specified, so server=8.8.8.8@1.2.3.4 or server=8.8.8.8@eth0 will use random source ports. server=8.8.8.8@1.2.3.4#66 or any use of query-port will use the explicitly configured port, and should only be done with understanding of the security implications. Note that this change changes non-existing interface, or non-local source address errors from fatal to run-time. The error will be logged and communication with the server not possible.
Change the method of allocation of random source ports for DNS. Previously, without min-port or max-port configured, dnsmasq would default to the compiled in defaults for those, which are 1024 and 65535. Now, when neither are configured, it defaults instead to the kernel’s ephemeral port range, which is typically 32768 to 60999 on Linux systems. This change eliminates the possibility that dnsmasq may be using a registered port > 1024 when a long-running daemon starts up and wishes to claim it. This change does likely slightly reduce the number of random ports and therefore the protection from reply spoofing. The older behavior can be restored using the min-port and max-port config switches should that be a concern.
Scale the size of the DNS random-port pool based on the value of the dns-forward-max configuration.
TFTP tweak: Check sender of all received packets, as specified in RFC 1350 para 4.
Brought up the docker container on a two different PCs (one running Debian and the other Ubuntu) with no pre-existing configuration at all and a very simple docker-compose.yml file and get the same error:
pihole | [âś—] DNS service is NOT listening
pihole | kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
docker-compose.yml
version: "3"
# More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/
services:
pihole:
restart: always
container_name: pihole
image: pihole/pihole:latest
ports:
- "53:53/tcp"
- "53:53/udp"
- "67:67/udp"
- "80:80/tcp"
- "443:443/tcp"
environment:
TZ: 'Europe/London'
WEBPASSWORD: 'xxxxxxxxxx'
# Volumes store your data between container upgrades
volumes:
- './etc-pihole/:/etc/pihole/'
- './etc-dnsmasq.d/:/etc/dnsmasq.d/'
# Recommended but not required (DHCP needs NET_ADMIN)
# https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
cap_add:
- NET_ADMIN
restart: unless-stopped
Do the exact same thing using 5.7 and it starts up fine:
pihole | [âś“] DNS service is listening
pihole | [âś“] UDP (IPv4)
pihole | [âś“] TCP (IPv4)
pihole | [âś“] UDP (IPv6)
pihole | [âś“] TCP (IPv6)
I stand corrected -- my docker logs pihole shows the same failure message that yours does.. but in my case, pihole is actually responding to DNS queries. So the logs indicate failure, but pihole is listening for DNS requests.
According to the FTL src, "pihole-FTL test" performs a full startup, and then immediately exits. This is to confirm that the configs are valid. Then the s6 20-start.sh continues:
-> if gravity is *NOT* skipped on boot ($SKIPGRAVITYONBOOT), then:
-> gravity.sh
-> pihole status
-> looks for something listening on port 53, if it's not found (or if it's not pihole):
-> echo -e " ${CROSS} DNS service is NOT listening"
Perhaps in pihole <= 5.7, pihole-FTL test would fork and remain running (and listening on :53) long enough for gravity.sh to complete, so it got a clean pihole status at the end. but now, the FTL test process is gone by the time gravity's status check looks for a listener on port 53. Hence the error output.
Then 20-start.sh continues:
-> # Kill dnsmasq because s6 won't like it if it's running when s6 services start
kill -9 $(pgrep pihole-FTL) || true # TODO: REVISIT THIS SO AS TO NOT kill -9
-> prints "kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]"
This prints the kill usage because pgrep didn't find a pihole-FTL process. More evidence that this startup script expected pihole-FTL test to still be running by the time we got here.
If you have deleted or commented out the IP addresses in the setupvars, pihole -g returns the following error message:
"No IP addresses found! Please run 'pihole -r' to reconfigure".
Managed to get pi-hole running. It was failing to resolve DNS requests and was hanging on startup at the 'DNS service is NOT listening' message but by doing the following I have managed to get it running, although it still reports that message in the logs it does actually run. Steps taken:
Stop v5.7 and download v5.8
Completely delete all my existing configuration
Start pi-hole container
Login to the UI and re-enter all my settings
EDIT: Actually ignore that. It worked briefly until a reboot and then it's stopped working again and hangs on startup. Back to v5.7 for me.
After a bit of digging around I've got v5.8 running in Docker. It still shows the error message in the logs on startup, but it does actually run and resolve DNS requests. There's a bit of detail required here, but it's something that could easily affect other people so it's worth going over.
My network has a gateway that hands out IP addresses, DNS settings etc via DHCP (pretty standard) so this is the DNS server for my network and it forwards on requests to Pi-hole.
I have a small low power PC that runs Pi-hole in a docker container. This got it's address from a static reservation in the DHCP server. So at this point you have the scenario where the Pi-hole system is using the DHCP assigned DNS servers which are in turn itself.
During startup of the docker container it is trying to do some queries to the internet (looking for newer versions etc) which cannot work because there is no DNS server running because Pi-hole is not running. This in turn stops the docker container from starting up (or at the very least stall for a very long time).
I resolved this by setting a static IP and DNS etc on the server running the docker container so there is a way for DNS requests to work on the startup of the docker container.
BUT - this setup worked fine prior to the release of v5.8, so I think the comment about the startup changing and previously 'pihole-ftl test' being left running is true and this was allowing the startup to run previously.
It can be argued that I should have used a static IP on the server before, but controlling IPs through a single place on the gateway with reservations is much simpler to manage and it did work previously.
Either the docker container should avoid doing queries to the internet on startup before it is able to serve DNS requests or this situation should be documented so people can avoid it.
I did the upgrade, and am not using docker, but several hours after the upgrade my setup went to "DNS service is NOT listening". I was able to get this "fixed" temporarily by changing my DNS settings and saving, but I have noticed lots of database failure messages in the FTL log, for example here:
s ON network_addresses.network_id = network.id WHERE network_addresses.ip = ? AND interface != 'N/A' AND interface IS NOT NULL;] data (11)
[2021-04-16 10:46:00.746 823/T827] SQLite3 message: database corruption at line 67162 of [5d4c65779d] (11)
[2021-04-16 10:46:00.746 823/T827] SQLite3 message: statement aborts at 3: [SELECT MAX(ID) FROM queries] database disk image is malformed (11)
[2021-04-16 10:46:00.746 823/T827] Encountered step error in get_max_query_ID(): database disk image is malformed
[2021-04-16 10:46:00.747 823/T827] Error while trying to close database: database is locked
[2021-04-16 10:46:00.747 823/T827] Encountered error while trying to store queries in long-term database: database is locked
[2021-04-16 10:46:00.767 823/T827] ERROR: SQL query "DELETE FROM network_addresses WHERE lastSeen < 1587033960;" failed: database is locked
[2021-04-16 10:46:00.768 823/T827] Error while trying to close database: database is locked
[2021-04-16 10:46:00.768 823/T827] Error while trying to close database: database is locked
[2021-04-16 10:46:37.966 823M] SQLite3 message: database corruption at line 81887 of [5d4c65779d] (11)
[2021-04-16 10:46:37.966 823M] SQLite3 message: statement aborts at 25: [SELECT hwaddr FROM network WHERE id = (SELECT network_id FROM network_addresses WHERE ip = ? GROUP BY ip HAVING max(lastSeen));] database disk image is malformed (11)
[2021-04-16 10:46:37.967 823M] SQLite3 message: database corruption at line 81887 of [5d4c65779d] (11)
[2021-04-16 10:46:37.967 823M] SQLite3 message: statement aborts at 9: [SELECT name FROM network_addresses WHERE name IS NOT NULL AND ip = ?;] database disk image is malformed (11)
[2021-04-16 10:46:37.968 823M] SQLite3 message: database corruption at line 81887 of [5d4c65779d] (11)
[2021-04-16 10:46:37.968 823M] SQLite3 message: statement aborts at 20: [SELECT name FROM network_addresses WHERE name IS NOT NULL AND network_id = (SELECT network_id FROM network_addresses WHERE ip = ?) ORDER BY lastSeen DESC LIMIT 1] database disk image i (11)
[2021-04-16 10:46:37.970 823M] SQLite3 message: database corruption at line 81887 of [5d4c65779d] (11)
[2021-04-16 10:46:37.970 823M] SQLite3 message: statement aborts at 10: [SELECT interface FROM network JOIN network_addresses ON network_addresses.network_id = network.id WHERE network_addresses.ip = ? AND interface != 'N/A' AND interface IS NOT NULL;] data (11)
[2021-04-16 10:47:00.846 823/T827] SQLite3 message: database corruption at line 67162 of [5d4c65779d] (11)
[2021-04-16 10:47:00.846 823/T827] SQLite3 message: statement aborts at 3: [SELECT MAX(ID) FROM queries] database disk image is malformed (11)
[2021-04-16 10:47:00.846 823/T827] Encountered step error in get_max_query_ID(): database disk image is malformed
[2021-04-16 10:47:00.846 823/T827] Error while trying to close database: database is locked
[2021-04-16 10:47:00.846 823/T827] Encountered error while trying to store queries in long-term database: database is locked
[2021-04-16 10:47:00.867 823/T827] ERROR: SQL query "DELETE FROM network_addresses WHERE lastSeen < 1587034020;" failed: database is locked
[2021-04-16 10:47:00.868 823/T827] Error while trying to close database: database is locked
[2021-04-16 10:47:00.868 823/T827] Error while trying to close database: database is locked