I've been running Pi-Hole v6 in a docker container since quite a while, and have only recently - i.e. after updating last night - into a bit of a pickle: after running the update, the pi-hole container isn't able to remain 'healthy' and DNS resolution by Pi-Hole does not work. When I divert my network's DNS queries to another server (Quad9, though that's irrelevant to the issue at hand) and I restart, Pi-Hole seams to get healthy for a bit but as soon as I redirect the DNS queries to Pi-Hole, the container becomes unhealthy again.
FTL log :
2025-01-04 10:02:29.220 INFO PID of FTL process: 49
2025-01-04 10:02:29.224 INFO listening on 0.0.0.0 port 53
2025-01-04 10:02:29.224 INFO listening on :: port 53
2025-01-04 10:02:29.226 INFO PID of FTL process: 49
2025-01-04 10:02:29.231 ERROR SQLite3: recovered 26 frames from WAL file /etc/pihole/pihole-FTL.db-wal (283)
2025-01-04 10:02:29.233 INFO Database version is 21
2025-01-04 10:02:29.244 INFO Database successfully initialized
2025-01-04 10:02:30.922 INFO Imported 20157 queries from the on-disk database (it has 602412 rows)
2025-01-04 10:02:30.922 INFO Parsing queries in database
2025-01-04 10:02:31.013 INFO 10000 queries parsed...
2025-01-04 10:02:31.087 INFO 20000 queries parsed...
2025-01-04 10:02:31.089 INFO Imported 20157 queries from the long-term database
2025-01-04 10:02:31.089 INFO -> Total DNS queries: 20157
2025-01-04 10:02:31.089 INFO -> Cached DNS queries: 1293
2025-01-04 10:02:31.089 INFO -> Forwarded DNS queries: 9107
2025-01-04 10:02:31.089 INFO -> Blocked DNS queries: 9480
2025-01-04 10:02:31.089 INFO -> Unknown DNS queries: 32
2025-01-04 10:02:31.089 INFO -> Unique domains: 1324
2025-01-04 10:02:31.089 INFO -> Unique clients: 2
2025-01-04 10:02:31.089 INFO -> DNS cache records: 869
2025-01-04 10:02:31.089 INFO -> Known forward destinations: 3
2025-01-04 10:02:31.395 INFO FTL is running as user pihole (UID 100)
2025-01-04 10:02:31.397 INFO Restored 0 API sessions from the database
2025-01-04 10:02:31.400 INFO Blocking status is enabled
2025-01-04 10:02:31.511 INFO Compiled 4 allow and 15 deny regex for 2 clients in 11.6 msec
2025-01-04 10:02:35.410 ERROR Cannot receive UDP DNS reply: Timeout - no response from upstream DNS server
2025-01-04 10:02:35.410 INFO Tried to resolve PTR "1.0.16.172.in-addr.arpa" on 127.0.0.1#53 (UDP)
2025-01-04 10:02:35.419 INFO Received 8/8 valid NTP replies from pool.ntp.org
2025-01-04 10:02:35.420 INFO Time offset: -4.482269e-02 ms (excluded 1 outliers)
2025-01-04 10:02:35.420 INFO Round-trip delay: 2.622604e-01 ms (excluded 1 outliers)
2025-01-04 10:02:41.776 ERROR Cannot receive UDP DNS reply: Timeout - no response from upstream DNS server
2025-01-04 10:02:41.777 INFO Tried to resolve PTR "3.0.16.172.in-addr.arpa" on 127.0.0.1#53 (UDP)
2025-01-04 10:02:46.032 ERROR Cannot receive UDP DNS reply: Timeout - no response from upstream DNS server
2025-01-04 10:02:46.032 INFO Tried to resolve PTR "112.112.112.149.in-addr.arpa" on 127.0.0.1#53 (UDP)
The last few lines look like they might have something to do with matters...
Some further info:
Running on a Firewalla Purple device (have done so for years)
Using Unbound as upstream DNS (but the issue persists if I configure Quad9 as upstream DNS in Pi-Hole as well)
Issue appears with both nightly (which I was running up until now) and development tags
I've checked if the 'upstream' Unbound container works, and a test shows it does
Interestingly, 1 hour later, some additional lines are added:
2025-01-04 10:15:38.227 INFO PID of FTL process: 55
2025-01-04 10:15:38.228 INFO listening on 0.0.0.0 port 53
2025-01-04 10:15:38.228 INFO listening on :: port 53
2025-01-04 10:15:38.229 INFO PID of FTL process: 55
2025-01-04 10:15:38.232 INFO Database version is 21
2025-01-04 10:15:38.234 INFO Database successfully initialized
2025-01-04 10:15:40.108 INFO Imported 20159 queries from the on-disk database (it has 602427 rows)
2025-01-04 10:15:40.108 INFO Parsing queries in database
2025-01-04 10:15:40.228 INFO 10000 queries parsed...
2025-01-04 10:15:40.299 INFO 20000 queries parsed...
2025-01-04 10:15:40.300 INFO Imported 20159 queries from the long-term database
2025-01-04 10:15:40.301 INFO -> Total DNS queries: 20159
2025-01-04 10:15:40.301 INFO -> Cached DNS queries: 1292
2025-01-04 10:15:40.301 INFO -> Forwarded DNS queries: 9110
2025-01-04 10:15:40.301 INFO -> Blocked DNS queries: 9480
2025-01-04 10:15:40.301 INFO -> Unknown DNS queries: 32
2025-01-04 10:15:40.301 INFO -> Unique domains: 1326
2025-01-04 10:15:40.301 INFO -> Unique clients: 2
2025-01-04 10:15:40.301 INFO -> DNS cache records: 869
2025-01-04 10:15:40.301 INFO -> Known forward destinations: 3
2025-01-04 10:15:40.590 INFO FTL is running as user pihole (UID 100)
2025-01-04 10:15:40.593 INFO Restored 0 API sessions from the database
2025-01-04 10:15:40.595 INFO Blocking status is enabled
2025-01-04 10:15:40.735 INFO Compiled 4 allow and 15 deny regex for 2 clients in 30.2 msec
2025-01-04 10:15:44.615 INFO Received 8/8 valid NTP replies from pool.ntp.org
2025-01-04 10:15:44.616 INFO Time offset: -2.390998e-02 ms (excluded 1 outliers)
2025-01-04 10:15:44.616 INFO Round-trip delay: 2.640315e-01 ms (excluded 1 outliers)
2025-01-04 10:15:44.624 ERROR Cannot receive UDP DNS reply: Timeout - no response from upstream DNS server
2025-01-04 10:15:44.624 INFO Tried to resolve PTR "1.0.16.172.in-addr.arpa" on 127.0.0.1#53 (UDP)
2025-01-04 10:15:53.264 ERROR Cannot receive UDP DNS reply: Timeout - no response from upstream DNS server
2025-01-04 10:15:53.264 INFO Tried to resolve PTR "3.0.16.172.in-addr.arpa" on 127.0.0.1#53 (UDP)
2025-01-04 10:16:01.328 ERROR Cannot receive UDP DNS reply: Timeout - no response from upstream DNS server
2025-01-04 10:16:01.328 INFO Tried to resolve PTR "9.9.9.9.in-addr.arpa" on 127.0.0.1#53 (UDP)
2025-01-04 11:15:47.226 INFO Wrote config file:
2025-01-04 11:15:47.228 INFO - 151 total entries
2025-01-04 11:15:47.229 INFO - 133 entries are default
2025-01-04 11:15:47.229 INFO - 18 entries are modified
2025-01-04 11:15:47.229 INFO - 0 entries are forced through environment
2025-01-04 11:15:47.239 INFO Config file written to /etc/pihole/pihole.toml
2025-01-04 11:15:48.661 INFO Received 8/8 valid NTP replies from '
2025-01-04 11:15:48.661 INFO Time offset: -2.639634e-02 ms (excluded 1 outliers)
2025-01-04 11:15:48.661 INFO Round-trip delay: 2.507482e-01 ms (excluded 1 outliers)
2025-01-04 11:35:45.232 ERROR Cannot receive UDP DNS reply: Timeout - no response from upstream DNS server
2025-01-04 11:35:45.233 INFO Tried to resolve PTR "156.32.168.192.in-addr.arpa" on 127.0.0.1#53 (UDP)
As before, container looks healthy, yet the moment I configure the pi-hole IP as the LAN DNS server, the container goes unhealthy again and the error about not being able to get a UDP DNS reply reoccur.
Don't want to be pushy - but could anyone pls have a quick look?
In the mean time, I've tried a couple of things.
The most important is, that if I set the Unbound IP as the DNS server address, everything works, so that is a slight indication that the issue is unlikely to be found outside of the Pi-Hole docker container.
In the mean time, I've added tried by adding the config of the DNSMASQ_USER: root etc.
I seems increasingly like the error that is found in the FTL log regarding UDP has something to do with what's causing all of this...
Currently I'm on the nightly branch again, as I had read about some changes and possible bugs in DNSMASQ, tell me if you want me to switch back to development please.
In pihole.log, I see this (taken from the web interface), the last two lines keep coming back.
No other requests are coming in for now, as I've switched to Quad9 to keep the family happy
2025-01-05 10:36:14.988 started, version pi-hole-v2.91test4 cache disabled
2025-01-05 10:36:14.988 compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n IDN2 DHCP DHCPv6 Lua TFTP no-conntrack ipset no-nftset auth DNSSEC loop-detect inotify dumpfile
2025-01-05 10:36:14.988 using nameserver 172.16.0.3#5335
2025-01-05 10:36:14.988 using nameserver 172.16.0.3#5335
2025-01-05 10:36:14.988 using only locally-known addresses for onion
2025-01-05 10:36:14.988 using only locally-known addresses for bind
2025-01-05 10:36:14.988 using only locally-known addresses for invalid
2025-01-05 10:36:14.988 using only locally-known addresses for localhost
2025-01-05 10:36:14.988 using only locally-known addresses for test
2025-01-05 10:36:14.988 using only locally-known addresses for lan
2025-01-05 10:36:14.988 read /etc/hosts - 9 names
2025-01-05 10:36:14.988 read /etc/pihole/hosts/custom.list - 3 names
2025-01-05 10:36:16.844 **query[A] firewalla.encipher.io from 172.16.0.1**
2025-01-05 10:36:19.203 forwarded firewalla.encipher.io to 172.16.0.3#5335
2025-01-05 10:36:19.203 forwarded firewalla.encipher.io to 172.16.0.3#5335
2025-01-05 10:36:19.203 reply firewalla.encipher.io is 34.214.149.62
2025-01-05 10:36:19.203 reply firewalla.encipher.io is 54.201.47.113
2025-01-05 10:36:19.203 reply firewalla.encipher.io is 52.34.214.185
2025-01-05 10:36:19.204 query[PTR] 1.0.16.172.in-addr.arpa from 127.0.0.1
2025-01-05 10:36:23.301 config 172.16.0.1 is NXDOMAIN
2025-01-05 10:36:23.302 query[PTR] 156.32.168.192.in-addr.arpa from 127.0.0.1
2025-01-05 10:36:27.351 config 192.168.32.156 is NXDOMAIN
2025-01-05 10:36:27.351 query[PTR] 1.0.0.127.in-addr.arpa from 127.0.0.1
2025-01-05 10:36:31.355 /etc/hosts 127.0.0.1 is localhost
2025-01-05 10:36:31.355 query[PTR] 193.32.168.192.in-addr.arpa from 127.0.0.1
2025-01-05 10:36:35.388 config 192.168.32.193 is NXDOMAIN
2025-01-05 10:36:35.388 query[PTR] 2.0.16.172.in-addr.arpa from 127.0.0.1
2025-01-05 10:36:39.444 config 2.0.16.172.in-addr.arpa is <PTR>
2025-01-05 10:36:39.444 **query[A] pi.hole from 127.0.0.1**
2025-01-05 10:36:39.445 Pi-hole hostname pi.hole is 0.0.0.0
2025-01-05 10:36:39.445 query[PTR] 135.32.168.192.in-addr.arpa from 127.0.0.1
2025-01-05 10:36:43.471 config 192.168.32.135 is NXDOMAIN
2025-01-05 10:36:43.471 query[PTR] 3.0.16.172.in-addr.arpa from 127.0.0.1
2025-01-05 10:36:47.493 config 172.16.0.3 is NXDOMAIN
2025-01-05 10:37:09.770 **query[A] pi.hole from 127.0.0.1**
2025-01-05 10:37:09.770 Pi-hole hostname pi.hole is 0.0.0.0
2025-01-05 10:37:40.071 **query[A] pi.hole from 127.0.0.1**
2025-01-05 10:37:40.071 Pi-hole hostname pi.hole is 0.0.0.0
I'll briefly switch the network DNS server back to Pi-Hole and send another view on the pihole log, but it does give the impression that resolution actually works (e.g. there is an address provided for the firewalla.encipher.io domain. I am gobsmacked and faced with the reality that I am an absolutely noob at this.
Here's an extract of pihole.log after configuring PI-Hole again as DNS server for the network, the last website I wanted to surf to was sonusfaber.com - for reference:
2025-01-05 11:12:29.747 Pi-hole hostname pi.hole is 0.0.0.0
2025-01-05 11:13:00.058 **query[A] pi.hole from 127.0.0.1**
2025-01-05 11:13:00.058 Pi-hole hostname pi.hole is 0.0.0.0
2025-01-05 11:13:30.302 **query[A] pi.hole from 127.0.0.1**
2025-01-05 11:13:30.302 Pi-hole hostname pi.hole is 0.0.0.0
2025-01-05 11:14:00.591 **query[A] pi.hole from 127.0.0.1**
2025-01-05 11:14:00.591 Pi-hole hostname pi.hole is 0.0.0.0
2025-01-05 11:14:03.178 **query[AAAA] my.tado.com from 172.16.0.1**
2025-01-05 11:14:05.903 forwarded my.tado.com to 172.16.0.3#5335
2025-01-05 11:14:05.903 forwarded my.tado.com to 172.16.0.3#5335
2025-01-05 11:14:05.904 reply my.tado.com is NODATA-IPv6
2025-01-05 11:14:08.158 **query[AAAA] my.tado.com from 172.16.0.1**
2025-01-05 11:14:08.159 forwarded my.tado.com to 172.16.0.3#5335
2025-01-05 11:14:08.159 reply my.tado.com is NODATA-IPv6
2025-01-05 11:14:14.181 **query[A] fcm.googleapis.com from 172.16.0.1**
2025-01-05 11:14:16.558 forwarded fcm.googleapis.com to 172.16.0.3#5335
2025-01-05 11:14:16.558 **query[A] fcm.googleapis.com from 172.16.0.1**
2025-01-05 11:14:16.558 forwarded fcm.googleapis.com to 172.16.0.3#5335
2025-01-05 11:14:16.558 forwarded fcm.googleapis.com to 172.16.0.3#5335
2025-01-05 11:14:16.560 query[HTTPS] www.sportsmaserati.com from 172.16.0.1
2025-01-05 11:14:19.245 **regex denied www.sportsmaserati.com is NODATA**
2025-01-05 11:14:19.248 reply fcm.googleapis.com is 216.239.38.57
2025-01-05 11:14:19.248 reply fcm.googleapis.com is 216.239.36.55
2025-01-05 11:14:19.248 reply fcm.googleapis.com is 216.239.38.55
2025-01-05 11:14:19.248 reply fcm.googleapis.com is 216.239.36.57
2025-01-05 11:14:19.248 query[HTTPS] www.sportsmaserati.com from 172.16.0.1
2025-01-05 11:14:19.248 **regex denied www.sportsmaserati.com is NODATA**
2025-01-05 11:14:19.248 **query[A] fcm.googleapis.com from 172.16.0.1**
2025-01-05 11:14:19.248 forwarded fcm.googleapis.com to 172.16.0.3#5335
2025-01-05 11:14:19.248 query[HTTPS] www.sportsmaserati.com from 172.16.0.1
2025-01-05 11:14:19.248 **regex denied www.sportsmaserati.com is NODATA**
2025-01-05 11:14:19.249 reply fcm.googleapis.com is 216.239.38.57
2025-01-05 11:14:19.249 reply fcm.googleapis.com is 216.239.36.55
2025-01-05 11:14:19.249 reply fcm.googleapis.com is 216.239.38.55
2025-01-05 11:14:19.249 reply fcm.googleapis.com is 216.239.36.57
2025-01-05 11:14:19.249 **query[A] firewalla.encipher.io from 172.16.0.1**
2025-01-05 11:14:19.249 forwarded firewalla.encipher.io to 172.16.0.3#5335
2025-01-05 11:14:19.249 query[HTTPS] sonusfaber.com from 172.16.0.1
2025-01-05 11:14:21.002 **regex denied sonusfaber.com is NODATA**
2025-01-05 11:14:21.002 reply firewalla.encipher.io is 54.201.47.113
2025-01-05 11:14:21.002 reply firewalla.encipher.io is 52.34.214.185
2025-01-05 11:14:21.002 reply firewalla.encipher.io is 34.214.149.62
2025-01-05 11:14:21.002 **query[A] sonusfaber.com from 172.16.0.1**
2025-01-05 11:14:22.729 forwarded sonusfaber.com to 172.16.0.3#5335
2025-01-05 11:14:22.729 **query[AAAA] sonusfaber.com from 172.16.0.1**
2025-01-05 11:14:24.453 forwarded sonusfaber.com to 172.16.0.3#5335
2025-01-05 11:14:24.454 reply sonusfaber.com is 76.76.21.21
2025-01-05 11:14:24.454 query[HTTPS] sonusfaber.com from 172.16.0.1
2025-01-05 11:14:24.454 **regex denied sonusfaber.com is NODATA**
2025-01-05 11:14:24.455 **query[AAAA] sonusfaber.com from 172.16.0.1**
2025-01-05 11:14:24.455 forwarded sonusfaber.com to 172.16.0.3#5335
2025-01-05 11:14:24.455 forwarded sonusfaber.com to 172.16.0.3#5335
2025-01-05 11:14:24.455 **query[A] sonusfaber.com from 172.16.0.1**
2025-01-05 11:14:24.455 forwarded sonusfaber.com to 172.16.0.3#5335
2025-01-05 11:14:24.455 **query[A] fcm.googleapis.com from 172.16.0.1**
2025-01-05 11:14:24.455 forwarded fcm.googleapis.com to 172.16.0.3#5335
2025-01-05 11:14:24.455 reply sonusfaber.com is 76.76.21.21
2025-01-05 11:14:24.455 query[HTTPS] www.sportsmaserati.com from 172.16.0.1
2025-01-05 11:14:24.455 **regex denied www.sportsmaserati.com is NODATA**
2025-01-05 11:14:24.456 reply fcm.googleapis.com is 216.239.38.55
2025-01-05 11:14:24.456 reply fcm.googleapis.com is 216.239.36.57
2025-01-05 11:14:24.456 reply fcm.googleapis.com is 216.239.38.57
2025-01-05 11:14:24.456 reply fcm.googleapis.com is 216.239.36.55
2025-01-05 11:14:24.456 **query[A] www.google.com from 172.16.0.1**
2025-01-05 11:14:26.762 forwarded www.google.com to 172.16.0.3#5335
2025-01-05 11:14:26.762 reply www.google.com is 142.250.179.164
2025-01-05 11:14:26.763 reply sonusfaber.com is NODATA-IPv6
2025-01-05 11:14:26.763 query[HTTPS] sonusfaber.com from 172.16.0.1
2025-01-05 11:14:26.763 **regex denied sonusfaber.com is NODATA**
2025-01-05 11:14:26.763 **query[AAAA] sonusfaber.com from 172.16.0.1**
2025-01-05 11:14:26.763 forwarded sonusfaber.com to 172.16.0.3#5335
2025-01-05 11:14:26.763 **query[A] sonusfaber.com from 172.16.0.1**
2025-01-05 11:14:26.763 forwarded sonusfaber.com to 172.16.0.3#5335
2025-01-05 11:14:26.763 reply sonusfaber.com is NODATA-IPv6
2025-01-05 11:14:26.763 **query[AAAA] config.edge.skype.com from 172.16.0.1**
2025-01-05 11:14:29.651 forwarded config.edge.skype.com to 172.16.0.3#5335
2025-01-05 11:14:29.667 reply config.edge.skype.com is <CNAME>
2025-01-05 11:14:33.730 reply config.edge.skype.com.trafficmanager.net is <CNAME>
2025-01-05 11:14:36.607 reply l-0007.config.skype.com is <CNAME>
2025-01-05 11:14:39.562 reply config-edge-skype.l-0007.l-msedge.net is <CNAME>
2025-01-05 11:14:41.922 reply l-0007.l-msedge.net is 2620:1ec:21::16
2025-01-05 11:14:41.922 reply sonusfaber.com is 76.76.21.21
2025-01-05 11:14:41.922 **query[A] config.edge.skype.com from 172.16.0.1**
2025-01-05 11:14:44.855 forwarded config.edge.skype.com to 172.16.0.3#5335
2025-01-05 11:14:44.855 reply config.edge.skype.com is <CNAME>
2025-01-05 11:14:48.934 reply config.edge.skype.com.trafficmanager.net is <CNAME>
2025-01-05 11:14:51.944 reply l-0007.config.skype.com is <CNAME>
2025-01-05 11:14:54.809 reply config-edge-skype.l-0007.l-msedge.net is <CNAME>
2025-01-05 11:14:57.203 reply l-0007.l-msedge.net is 13.107.42.16
Sorry, maybe I'm being blind here but I'm confused now. Let's put the "unhealthy" state at the side right now, but everything in your very last pihole.log excerpt looks perfectly fine? Where is the "DNS resolution not working" part here?...
That's the bit where it becomes very confusing and unclear for me too...
The moment I switch the DNS settings back to the Pi-Hole IP address, none of the devices on the network get DNS 'responses'.
I'll do it just now and make a couple of screenshots.
Just to add: this isn't a new set-up, I've been running Pi-Hole (later on with the addition of Unbound) in a docker container on this very device for literally (3-4) years, and only since I updated things 2 days ago, have things stopped working... It had been quite a while since I had run an update as things were working nicely, and I just did it as I had a bit of time and wanted to bring things a bit closer to latest and greatest...
I set Pi-Hole 's IP (2x) as the DNS server in the firewall/router settings
I tried reaching tesla.com and it predictably didn't work
on a hunch, I though I could try and update Gravity - that worked and afterwards, resolution started working again and the devices all can access things again like before. This is literally all I did to make things work again.
(FWIW, I've been trying all kinds of things over the last 2 days, except simply updating gravity as I didn't see a link)
Does this make sense at all? Can I do anything to help to further provide info - in case this was some kind of weird combination of things that makes Pi-Hole (in Docker?) go bork? Or was one of the db's corrupted?