DNSMasq Has Stopped Working

PWD · June 2, 2025, 7:17am

Please follow the below template, it will help us to help you!

If you are Experiencing issues with a Pi-hole install that has non-standard elements (e.g you are using `nginx`, `apache2` or another reverse proxy, or there is some other aspect of your install that is customised) - please use the Community Help category.

Expected Behaviour:

Within the settings, you can set the DNSMasq and I have, mine reads

server=/met.no/8.8.8.8
server=/met.no/8.8.4.4 
server=/debian.org/8.8.8.8
server=/debian.org/8.8.4.4

:

Raspberry Pi OS (Debian Bookworm)
Raspberry Pi Zero 2 W
N/A
N/A

Actual Behaviour:

Since 6.1 it's not forwarding said requests.

Debug Token:

https://tricorder.pi-hole.net/UX417d5r/

Bucking_Horn · June 2, 2025, 7:50am

I cannot reproduce your observation.
My dockered Pi-hole will use those extra server lines as configured:

2025-06-02 09:37:54.160 query met.no from 172.17.0.1
2025-06-02 09:37:54.171 forwarded met.no to 8.8.8.8
2025-06-02 09:37:54.192 reply met.no is 157.249.120.35
2025-06-02 09:37:54.192 reply met.no is 157.249.121.84

This also works with DNSSEC enabled (click for details).

2025-06-02 10:05:02.688 query met.no from 172.17.0.1
2025-06-02 10:05:02.710 forwarded met.no to 8.8.8.8
2025-06-02 10:05:02.756 dnssec-query[DS] no to 8.8.8.8
2025-06-02 10:05:02.771 reply no is DS for keytag 38032, algo 13, digest 2
2025-06-02 10:05:02.771 dnssec-query[DS] met.no to 8.8.8.8
2025-06-02 10:05:02.791 dnssec-query[DNSKEY] no to 8.8.8.8
2025-06-02 10:05:02.813 reply no is DNSKEY keytag 38032, algo 13
2025-06-02 10:05:02.813 reply no is DNSKEY keytag 63968, algo 13
2025-06-02 10:05:02.816 reply met.no is DS for keytag 18984, algo 10, digest 2
2025-06-02 10:05:02.816 dnssec-query[DNSKEY] met.no to 8.8.8.8
2025-06-02 10:05:02.830 reply met.no is truncated
2025-06-02 10:05:02.832 dnssec-query[DNSKEY] met.no to 8.8.8.8
2025-06-02 10:05:02.852 reply met.no is DNSKEY keytag 18984, algo 10
2025-06-02 10:05:02.852 reply met.no is DNSKEY keytag 488, algo 10
2025-06-02 10:05:02.853 validation result is SECURE
2025-06-02 10:05:02.853 reply met.no is 157.249.121.84
2025-06-02 10:05:02.853 reply met.no is 157.249.120.35

However, your debug log shows quite a few lines with the following message:

WARNING: Connection error (8.8.8.8#53): 
TCP connection failed while receiving payload length from upstream 
(Resource temporarily unavailable)

Those appear for 8.8.4.4 also, and your debug log shows those messages also to be logged in Pi-hole diagnosis.

This would suggest that your chosen upstreams have been temporarily unavailable or inaccessible.

Do you still observe those?
If so, would switching to another set of upstreams fix your issue?
What does dig @8.8.8.8 met.no return?

PWD · June 2, 2025, 8:12am

dig @8.8.8.8 met.no

; <<>> DiG 9.18.33-1~deb12u2-Debian <<>> @8.8.8.8 met.no
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46778
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;met.no.                                IN      A

;; ANSWER SECTION:
met.no.                 3600    IN      A       157.249.120.35
met.no.                 3600    IN      A       157.249.121.84

;; Query time: 67 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Mon Jun 02 09:10:31 BST 2025
;; MSG SIZE  rcvd: 67

PWD · June 2, 2025, 10:33am

I tried to see if 8.8.4.4 wasn't responding

dig 8.8.4.4

; <<>> DiG 9.18.33-1~deb12u2-Debian <<>> 8.8.4.4
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 41714
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;8.8.4.4.                       IN      A

;; AUTHORITY SECTION:
.                       86399   IN      SOA     a.root-servers.net. nstld.verisign-grs.com. 2025060200 1800 900 604800 86400

;; Query time: 27 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Mon Jun 02 11:30:53 BST 2025
;; MSG SIZE  rcvd: 111

PWD · June 3, 2025, 9:35am

So I changed debian.org to use 1.1.1.1 and it's still failing

PWD · June 6, 2025, 7:03am

Anyone?

Bucking_Horn · June 6, 2025, 10:55am

~~This may be similar to https://discourse.pi-hole.net/t/etc-dnsmasq-d-config-not-loading-after-updating-to-6-1/80309, though your forwards are for public domains rather than internal ones.~~ ~~It would seem that for forwards to work, there must be at least one entry in *Conditional Forwarding* enabled.~~ ~~To help you design such an entry, please upload a fresh debug log and share a fresh token.~~

EDIT: I was wrong here: Forwarding to public DNS servers is operational, also with the most recent FTL v6.2.2.

PWD · June 6, 2025, 11:28am

https://tricorder.pi-hole.net/3fOAaBK8/

PWD · June 7, 2025, 8:40am

https://tricorder.pi-hole.net/rVqmrhoG/

The latest debug after the latest update. The conditional forwarding of the DNSMasq still isn't working.

Bucking_Horn · June 8, 2025, 8:46am

As your forwards are to public DNS servers, you don't seem to be affected by etc_dnsmasq_d Config not loading after updating to 6.1 :
Using the most recent FTL v6.2.2., I can still successfully apply your configuration.

Your debug log shows the same temporary inaccessiblity for 1.1.1.1 as for 8.8.8.8:

WARNING: Connection error (1.0.0.1#53): 
TCP connection failed while receiving payload length from upstream 
(Resource temporarily unavailable)

Rather than sending a request to 8.8.4.4, that's a lookup for a domain with the name 8.8.4.4, which doesn't exist (NXDOMAIN).

Your dig @8.8.8.8 met.no output demonstrates that generally resolution via 8.8.8.8 is working.

Did you run that from your Pi-hole machine?

Could you share the output of the following digs, as run from your Pi-hole machine?

dig @8.8.8.8 met.no

dig @192.168.0.167 met.no

PWD · June 8, 2025, 9:11am

I ssh'd in, but yeah

Sure.
Someone (me) just learned how to quote!

dig @8.8.8.8 met.no

; <<>> DiG 9.18.33-1~deb12u2-Debian <<>> @8.8.8.8 met.no
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21124
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;met.no.                                IN      A

;; ANSWER SECTION:
met.no.                 3600    IN      A       157.249.116.253

;; Query time: 59 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Sun Jun 08 10:03:35 BST 2025
;; MSG SIZE  rcvd: 51

dig @192.168.0.167 met.no
;; communications error to 192.168.0.167#53: timed out
;; communications error to 192.168.0.167#53: timed out
;; communications error to 192.168.0.167#53: timed out

; <<>> DiG 9.18.33-1~deb12u2-Debian <<>> @192.168.0.167 met.no
; (1 server found)
;; global options: +cmd
;; no servers could be reached

Bucking_Horn · June 9, 2025, 9:19am

I guess those timeouts showed up as Resource temporarily unavailable in /var/log/pihole/pihole.log?

Just to be sure: 192.168.0.167 is your Pi-hole machine's IP?

PWD · June 9, 2025, 9:37am

Yep.

Yep, both Pi-Hole and Unbound are both on this IP

Bucking_Horn · June 9, 2025, 10:17am

Since I cannot reproduce your issue with my Pi-hole container, that would suggest that your observation is external to Pi-hole.

But since a direct dig from your Pi-hole machine to 8.8.8.8 succeeds, I've a hard time coming up with a potential explanation.
You wouldn't run any firewall on your Pi-hole machine that would prevent specifically Pi-hole's process to communicate with upstreams other than unbound?

PWD · June 9, 2025, 10:43am

I genuinely wouldn't even know how to. Also it's literally only blocking met.no and Debian.org

Bucking_Horn · June 9, 2025, 10:51am

Pi-hole is not blocking them - it's forwarding them to a public upstream as configured, but it would seem it is unable to communicate with those upstreams, reporting them as Resource temporarily unavailable.

Your debug log has expired - do you have DNSSEC enabled in Pi-hole?
If so, would disabling DNSSEC in Pi-hole affect your observation?

PWD · June 9, 2025, 11:13am

Standard (With DNSSEC): https://tricorder.pi-hole.net/peu0KF8V/

Without DNSSEC: https://tricorder.pi-hole.net/UcarpSgm/

Disabling DNSSEC has Debian working again. Finally we have a thread to pull on. Though I have no idea what this means or what the solution is.

Bucking_Horn · June 9, 2025, 11:38am

@DL6ER, I am loosely aware that dnsmasq may recently have introduced some changes to improve DNSSEC handling for domains forwarded to a particular server, probably as inspired by the discussion from [Dnsmasq-discuss] DNSSEC in dnsmasq's parent zone.
In the course of that discussion, Simon Kelley mentions a patch.

If that patch (or perhaps any other DNSSEC related one) would have been included in Pi-hole's currently embedded dnsmasq version, could that contribute to PWD's observation of the following custom lines to fail when DNSSEC is enabled?

server=/debian.org/8.8.8.8
server=/debian.org/8.8.4.4

A dig debian.org would then show up in pihole.log as e.g.

WARNING: Connection error (8.8.8.8#53): 
TCP connection failed while receiving payload length from upstream 
(Resource temporarily unavailable)

DL6ER · June 9, 2025, 5:43pm

I tried it myself by adding the two suggested server=/debian.org/8.8.{8.8|4.4} lines to my dnsmasq config and did the same dig:

$ dig @127.0.0.1 debian.org

; <<>> DiG 9.18.30-0ubuntu0.24.04.2-Ubuntu <<>> @127.0.0.1 debian.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56086
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;debian.org.                    IN      A

;; ANSWER SECTION:
debian.org.             225     IN      A       151.101.66.132
debian.org.             225     IN      A       151.101.194.132
debian.org.             225     IN      A       151.101.130.132
debian.org.             225     IN      A       151.101.2.132

;; Query time: 30 msec
;; SERVER: 127.0.0.1#53(127.0.0.1) (UDP)
;; WHEN: Mon Jun 09 19:26:38 CEST 2025
;; MSG SIZE  rcvd: 103

Jun  9 19:26:38 dnsmasq[1921741]: UDP 231 127.0.0.1/46950 query debian.org from 127.0.0.1
Jun  9 19:26:38 dnsmasq[1921741]: UDP 231 127.0.0.1/46950 forwarded debian.org to 8.8.8.8
Jun  9 19:26:38 dnsmasq[1921741]: UDP 231 127.0.0.1/46950 forwarded debian.org to 8.8.4.4
Jun  9 19:26:38 dnsmasq[1921741]: UDP 232 dnssec-query[DS] debian.org to 8.8.4.4
Jun  9 19:26:38 dnsmasq[1921741]: UDP 232 reply debian.org is DS for keytag 20225, algo 8, digest 2
Jun  9 19:26:38 dnsmasq[1921741]: UDP 233 dnssec-query[DNSKEY] debian.org to 8.8.4.4
Jun  9 19:26:38 dnsmasq[1921741]: UDP 233 reply debian.org is truncated
Jun  9 19:26:38 dnsmasq[2774137]: TCP 233 dnssec-query[DNSKEY] debian.org to 8.8.4.4
Jun  9 19:26:38 dnsmasq[2774137]: TCP 233 reply debian.org is DNSKEY keytag 40756, algo 8
Jun  9 19:26:38 dnsmasq[2774137]: TCP 233 reply debian.org is DNSKEY keytag 21715, algo 8
Jun  9 19:26:38 dnsmasq[2774137]: TCP 233 reply debian.org is DNSKEY keytag 20225, algo 8
Jun  9 19:26:38 dnsmasq[2774137]: TCP 233 reply debian.org is DNSKEY keytag 6004, algo 8
Jun  9 19:26:38 dnsmasq[1921741]: UDP 231 127.0.0.1/46950 validation result is SECURE
Jun  9 19:26:38 dnsmasq[1921741]: UDP 231 127.0.0.1/46950 reply debian.org is 151.101.66.132 (DNSSEC signed)
Jun  9 19:26:38 dnsmasq[1921741]: UDP 231 127.0.0.1/46950 reply debian.org is 151.101.194.132 (DNSSEC signed)
Jun  9 19:26:38 dnsmasq[1921741]: UDP 231 127.0.0.1/46950 reply debian.org is 151.101.130.132 (DNSSEC signed)
Jun  9 19:26:38 dnsmasq[1921741]: UDP 231 127.0.0.1/46950 reply debian.org is 151.101.2.132 (DNSSEC signed)

The DNSKEY for debian.org is too large to fit into a single UDP packet so we retry this specific query over TCP. Without DNSSEC, this will not happen as no DNSKEY will ever be requested.

So my assumption is that DNS via TCP is somehow an issue on your system, I try to simulate this by droping any TCP traffic on port 53 from my machine to the Internet:

sudo iptables -A OUTPUT -p tcp -m tcp --dport 53 -j DROP

and there we go:

$ dig @127.0.0.1 debian.org
;; communications error to 127.0.0.1#53: timed out
;; communications error to 127.0.0.1#53: timed out
;; communications error to 127.0.0.1#53: timed out

; <<>> DiG 9.18.30-0ubuntu0.24.04.2-Ubuntu <<>> @127.0.0.1 debian.org
; (1 server found)
;; global options: +cmd
;; no servers could be reached

and the log has:

Jun  9 19:31:23 dnsmasq[2778144]: UDP 7 127.0.0.1/40109 query debian.org from 127.0.0.1
Jun  9 19:31:23 dnsmasq[2778144]: UDP 7 127.0.0.1/40109 forwarded debian.org to 8.8.8.8
Jun  9 19:31:23 dnsmasq[2778144]: UDP 7 127.0.0.1/40109 forwarded debian.org to 8.8.4.4
Jun  9 19:31:23 dnsmasq[2778144]: UDP 8 dnssec-query[DS] debian.org to 8.8.8.8
Jun  9 19:31:23 dnsmasq[2778144]: UDP 8 reply debian.org is DS for keytag 20225, algo 8, digest 2
Jun  9 19:31:23 dnsmasq[2778144]: UDP 9 dnssec-query[DNSKEY] debian.org to 8.8.8.8
Jun  9 19:31:24 dnsmasq[2778144]: UDP 9 reply debian.org is truncated
Jun  9 19:31:24 dnsmasq[2778160]: TCP 9 dnssec-query[DNSKEY] debian.org to 8.8.8.8
[...]
Jun  9 19:31:28 dnsmasq[2778144]: UDP 100 127.0.0.1/34915 query debian.org from 127.0.0.1
[...]
Jun  9 19:31:30 dnsmasq[2778165]: TCP connection failed: Operation now in progress
[...]
Jun  9 19:31:33 dnsmasq[2778144]: UDP 150 127.0.0.1/60123 query debian.org from 127.0.0.1
Jun  9 19:31:33 dnsmasq[2778144]: UDP 150 127.0.0.1/60123 forwarded debian.org to 8.8.8.8
Jun  9 19:31:33 dnsmasq[2778144]: UDP 151 dnssec-query[DNSKEY] debian.org to 8.8.8.8
Jun  9 19:31:33 dnsmasq[2778144]: UDP 151 reply debian.org is truncated
Jun  9 19:31:34 dnsmasq[2778191]: TCP 151 dnssec-query[DNSKEY] debian.org to 8.8.8.8
[...]
Jun  9 19:31:35 dnsmasq[2778175]: TCP connection failed: Operation now in progress

Okay, this was the wrong error so your issue is not with sending but receiving from upstream, so let's add

sudo iptables -A INPUT -p tcp -m tcp --sport 53 -j DROP

(note the subtle but important change from dport to sport rule here):

$ dig @127.0.0.1 debian.org
;; communications error to 127.0.0.1#53: timed out
;; communications error to 127.0.0.1#53: timed out
;; communications error to 127.0.0.1#53: timed out

; <<>> DiG 9.18.30-0ubuntu0.24.04.2-Ubuntu <<>> @127.0.0.1 debian.org
; (1 server found)
;; global options: +cmd
;; no servers could be reached

and, indeed,

Jun  9 19:37:52 dnsmasq[2783232]: UDP 7 127.0.0.1/54218 query debian.org from 127.0.0.1
Jun  9 19:37:52 dnsmasq[2783232]: UDP 7 127.0.0.1/54218 forwarded debian.org to 8.8.8.8
Jun  9 19:37:52 dnsmasq[2783232]: UDP 7 127.0.0.1/54218 forwarded debian.org to 8.8.4.4
Jun  9 19:37:52 dnsmasq[2783232]: UDP 8 dnssec-query[DS] debian.org to 8.8.8.8
Jun  9 19:37:52 dnsmasq[2783232]: UDP 8 reply debian.org is DS for keytag 20225, algo 8, digest 2
Jun  9 19:37:52 dnsmasq[2783232]: UDP 9 dnssec-query[DNSKEY] debian.org to 8.8.8.8
Jun  9 19:37:52 dnsmasq[2783232]: UDP 9 reply debian.org is truncated
Jun  9 19:37:52 dnsmasq[2783248]: TCP 9 dnssec-query[DNSKEY] debian.org to 8.8.8.8
[...]
Jun  9 19:37:57 dnsmasq[2783232]: UDP 152 127.0.0.1/49952 query debian.org from 127.0.0.1
[...]
Jun  9 19:38:08 dnsmasq[2783248]: TCP connection failed: Resource temporarily unavailable
[...]
Jun  9 19:38:23 dnsmasq[2783248]: TCP connection failed: Resource temporarily unavailable

So your issue seems to be with receiving DNS TCP replies.

@PWD Please test

dig @8.8.8.8 +tcp met.no

PWD · June 9, 2025, 6:02pm

dig @8.8.8.8 +tcp met.no

; <<>> DiG 9.18.33-1~deb12u2-Debian <<>> @8.8.8.8 +tcp met.no
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10195
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;met.no.                                IN      A

;; ANSWER SECTION:
met.no.                 1676    IN      A       157.249.116.253

;; Query time: 23 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (TCP)
;; WHEN: Mon Jun 09 18:59:00 BST 2025
;; MSG SIZE  rcvd: 51

This is with DNSSEC still off.