Having trouble on secondary pi-hole DNS resolution

olivernz · June 29, 2021, 5:28am

I have 2 piholes. My main one is a LXC container on my server works like a charm. So if I bounce the server I have a backup pihole (to mitigate sons and wife staging mutiny) so I have one running on a OrangePi. Worked 100% until something fried on the board. So replaced it with a OrangePi Zero2 (Armbian buster). Since then it won't quite work. I have a network controlled by a Unifi UDM Pro and it spans across several VLANs but all that should not materially involve this issue.

Pihole is set up to use coludflare DoH service on same OPi on 5353.

What I see on pihole 192.168.88.8:

dig @127.0.0.1 google.com

; <<>> DiG 9.16.1-Ubuntu <<>> @127.0.0.1 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37103
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             167     IN      A       142.250.66.238

;; Query time: 196 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Jun 29 17:17:48 NZST 2021
;; MSG SIZE  rcvd: 65

and:

dig @192.168.88.8 google.com

; <<>> DiG 9.16.1-Ubuntu <<>> @192.168.88.8 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47815
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             51      IN      A       142.250.66.238

;; Query time: 36 msec
;; SERVER: 192.168.88.8#53(192.168.88.8)
;; WHEN: Tue Jun 29 17:19:44 NZST 2021
;; MSG SIZE  rcvd: 55

But from 192.168.89.57 (sorry Windowze):

nslookup google.com 192.168.88.8
DNS request timed out.
    timeout was 2 seconds.
Server:  UnKnown
Address:  192.168.88.8

DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to UnKnown timed-out

And from 192.168.88.7:

dig @192.168.88.8 google.com

; <<>> DiG 9.16.1-Ubuntu <<>> @192.168.88.8 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16985
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             187     IN      A       142.250.66.238

;; Query time: 47 msec
;; SERVER: 192.168.88.8#53(192.168.88.8)
;; WHEN: Tue Jun 29 17:22:09 NZST 2021
;; MSG SIZE  rcvd: 65

There are several more machines it works from and some that it doesn't. There ar eno FW blocking traffic or anything like that. tcpdump shows traffic coming in on port 53 and i cannot see any other contending service on any of the involved ports.

tcpdump -i eth0 port 53
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
16:50:43.384574 IP 192.168.89.103.61325 > pihole.domain: 42231+ A? corp.bank.nzpfs.co.nz. (39)
16:50:43.387913 IP pihole.44271 > pihole-1.domain: 4234+ PTR? 103.89.168.192.in-addr.arpa. (45)
16:50:43.389544 IP pihole-1.domain > pihole.44271: 4234 NXDomain 0/0/1 (56)
16:50:43.392536 IP pihole.37681 > pihole-1.domain: 17981+ PTR? 8.88.168.192.in-addr.arpa. (43)
16:50:43.392988 IP pihole-1.domain > pihole.37681: 17981* 1/0/0 PTR pihole. (63)
16:50:43.394124 IP pihole.59720 > pihole-1.domain: 56339+ PTR? 88.88.168.192.in-addr.arpa. (44)
16:50:43.394530 IP pihole-1.domain > pihole.59720: 56339* 1/0/0 PTR pihole-1. (66)
16:50:45.392417 IP 192.168.89.103.61326 > pihole.domain: 50198+ A? corp.bank.nzpfs.co.nz. (39)
16:50:45.848480 IP 192.168.89.103.64566 > pihole.domain: 23497+ A? community.dynatrace.com. (41)
16:50:52.326579 IP 192.168.89.57.52229 > pihole.domain: 1+ PTR? 8.88.168.192.in-addr.arpa. (43)
16:50:52.327580 IP pihole.42748 > pihole-1.domain: 7774+ PTR? 57.89.168.192.in-addr.arpa. (44)
16:50:52.328850 IP pihole-1.domain > pihole.42748: 7774 NXDomain 0/0/1 (55)
16:50:54.334020 IP 192.168.89.57.53189 > pihole.domain: 2+ A? pihole.home. (29)
16:50:56.335253 IP 192.168.89.57.53190 > pihole.domain: 3+ AAAA? pihole.home. (29)
16:51:04.413431 IP defiant88.48885 > pihole.domain: 57685+ [1au] A? google.de. (50)

Here is debug link. I couldn't find anything obvious.
https://tricorder.pi-hole.net/zoklksweqh

This is doing my head in. I hope someone can point me in a direction I can have a look. Thanks heaps in advance!

oliver

Bucking_Horn · June 29, 2021, 10:05am

In general, you should avoid using port 5353, as that port is reserved for use by the mDNS protocol.

Note that in your specific case, according to your debug log, cloudflare already listens on a different port (5053), so this shouldn't be an issue in your installation.

Above would suggest that the DNS request has reached a host at 192.168.88.8 (your Pi-hole, according to your debug log), but either the reply was suppressed, or the request didn't make it to Pi-hole at all (otherwise, Server should have read pi.hole).
You should be able to confirm the former if Pi-hole's Query log would have registered that request.

It certainly has - clients in a specific VLAN can only communicate with your Pi-hole in another VLAN if either inter-VLAN routing is properly configured on involved routers and switches or if your Pi-hole host would have a valid connection (including an IP address) to each of the VLANs it's supposed to provide DNS services for.

As your eth0 interface carries only one IP address, I'm going to assume that you are using the former approach. In that case, you have configured Pi-hole to only listen for same subnet requests on interface eth0:

*** [ DIAGNOSING ]: Setup variables
    PIHOLE_INTERFACE=eth0

You should try changing Pi-hole's Interface listening behaviour (under Settings|DNS) to Listen on all interfaces, permit all origins.

If that doesn't help, I'd suspect your network's routing of DNS traffic needs to be adjusted as well, or a firewall on your Pi-hole host is still interfering somehow.

olivernz · June 29, 2021, 12:15pm

Thanks for that Bucking_Horn.

Yes, 5053. My bad there.

The VLANs can all talk to each other and I have clients from each VLAN that can connect and some that can't. Even in the own VLAN this the case.

I can see the request hit the OrangePi on tcpdump but I can't see a log in pihole. So it must be lost somewhere in-between. But I am not running a FW on the OrangePi (I checked). And even if, then I'd expect all queries to fail or there be a discernible pattern.

Lol. It's always the last point isn't it. A liitle different to what you said but you hit the nail on the head.

As you can see I didn't read the fine-print. It says "only one hop away". That would explain the erratic behaviour. Changed to the bottom option "Listen on all interfaces, permit all origins" and that made it all run like a charm! THATK YOU! And of course duh

Now, I looked on my other pihole and there it was set correctly. When I set up the new pihole I teleported the setup but I guess that wasn't transferred and I didn't think to check. Maybe there's some reasoning behind that (and maybe I changed it and can't remember) but may be worth a look and see if that should be pulled over or not.

Anyway thanks heaps for the second set of "eyes".

P.S. Where would I see a log message that those are being dropped? I read all logs but didn't see anything. Or does it only appear on debug level?

Bucking_Horn · July 1, 2021, 6:48am

If Pi-hole's supply would have been suppressed or lost somehow, than a query would have registered in the Query log, while the client's nslookup still would have timed out.

In your case, Pi-hole never received the request, as it wasn't listening, which matches your Query log not showing anything of it.

system · July 22, 2021, 6:48am

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.