Sudden increase in local PTR queries from localhost/PiHole itself

mrc989 · September 2, 2024, 9:32pm

Expected Behaviour:

I have PiHole running on a Raspberry Pi acting as both DNS and DHCP server.
I also have configured Unbound as per PiHole guide.
I have several OpenWrt APs around the house, with DHCP disabled for the relevant interfaces and custom DNS configured to point to the PiHole at 192.168.5.20.
Up until yesterday night I had around 5.000 queries per hour, which sounded normal. Many of the queries were reported from PiHole as coming from the IP of OpenWrt APs and I wanted to see the actual IPs making the queries so I tried editing the configuration.

Actual Behaviour:

Since many queries were reported as coming from the IP address of APs, yesterday evening I tried configuring a couple of OpenWrt APs by setting DHCP-option as "6,192.168.5.20", where 192.168.5.20 is the IP of the PiHole - as I read here: How do I configure my devices to use Pi-hole as their DNS server?.
I applied this configuration to APs at 192.168.5.2 and 192.168.5.3 even though, since DHCP is disabled on those APs, the applied configuration shouldn't be relevant at all.
Even though I am not sure it is important, I add that at the same time and on the same 2 APs I have disabled the "rules" for forcing DNS queries to PiHole (as explained here Force All DNS Queries Through PiHole with OpenWRT) which I had created yesterday morning since they were interfering with some devices that wouldn't work any longer.

TL,DR

Long story short, up until yesterday night I had around 5.000 queries per hour, now I get around 25.000, most of which (around 80%) are PTR queries to local IP addresses.

I have attached the debug token below. Please find also a couple of extracts from Tcpdump and PiHole log that may be relevant as well.

Debug Token:

https://tricorder.pi-hole.net/tiGujYu3/

It looks like the PiHole (localhost or 127.0.0.1) is querying itself to get PTR of my whole local network. The PiHole replies with several NXDOMAIN as many devices are not really existing on the network. Most queries (around 120.191 as of now) are directed to 192.168.5.88 which is not in use (it is even out of the DHCP range of PiHole...).

I have read all possible posts here and on other forums. I have the feeling that there may be a DNS loop even though I do not understand how that could have happened. I have no conditional forwarding configured.

If I can do some other tests to provide additional info, I'd be happy to do so.

TCP dump of loopback interface on port 53:

sudo tcpdump -i lo port 53

3:15:46.149085 IP localhost.localdomain.47314 > localhost.localdomain.domain: 32341+ [1au] PTR? 146.5.168.192.in-addr.arpa. (67)
23:15:46.149408 IP localhost.localdomain.domain > localhost.localdomain.47314: 32341 NXDomain* 0/0/1 (55)
23:15:46.200524 IP localhost.localdomain.37605 > localhost.localdomain.domain: 2057+ [1au] PTR? 88.5.168.192.in-addr.arpa. (66)
23:15:46.200857 IP localhost.localdomain.domain > localhost.localdomain.37605: 2057 NXDomain* 0/0/1 (54)
23:15:46.239621 IP localhost.localdomain.49379 > localhost.localdomain.domain: 44593+ [1au] PTR? 161.5.168.192.in-addr.arpa. (67)
23:15:46.239941 IP localhost.localdomain.domain > localhost.localdomain.49379: 44593 NXDomain* 0/0/1 (55)
23:15:46.288999 IP localhost.localdomain.54808 > localhost.localdomain.domain: 15341+ [1au] PTR? 141.5.168.192.in-addr.arpa. (67)
23:15:46.289331 IP localhost.localdomain.domain > localhost.localdomain.54808: 15341 NXDomain* 0/0/1 (55)
23:15:46.338937 IP localhost.localdomain.54060 > localhost.localdomain.domain: 33620+ [1au] PTR? 186.5.168.192.in-addr.arpa. (67)
23:15:46.339325 IP localhost.localdomain.domain > localhost.localdomain.54060: 33620 NXDomain* 0/0/1 (55)
23:15:46.392863 IP localhost.localdomain.35495 > localhost.localdomain.domain: 15322+ [1au] PTR? 146.5.168.192.in-addr.arpa. (67)
23:15:46.393245 IP localhost.localdomain.domain > localhost.localdomain.35495: 15322 NXDomain* 0/0/1 (55)
23:15:46.442835 IP localhost.localdomain.33256 > localhost.localdomain.domain: 61897+ [1au] PTR? 107.5.168.192.in-addr.arpa. (67)
23:15:46.443165 IP localhost.localdomain.domain > localhost.localdomain.33256: 61897 NXDomain* 0/0/1 (55)
23:15:46.491923 IP localhost.localdomain.39249 > localhost.localdomain.domain: 19269+ [1au] PTR? 174.5.168.192.in-addr.arpa. (67)
23:15:46.492249 IP localhost.localdomain.domain > localhost.localdomain.39249: 19269 NXDomain* 0/0/1 (55)
23:15:46.544431 IP localhost.localdomain.58129 > localhost.localdomain.domain: 4740+ [1au] PTR? 88.5.168.192.in-addr.arpa. (66)
23:15:46.544794 IP localhost.localdomain.domain > localhost.localdomain.58129: 4740 NXDomain* 0/0/1 (54)
23:15:46.594340 IP localhost.localdomain.46599 > localhost.localdomain.domain: 38704+ [1au] PTR? 131.5.168.192.in-addr.arpa. (67)
23:15:46.594706 IP localhost.localdomain.domain > localhost.localdomain.46599: 38704 NXDomain* 0/0/1 (55)
23:15:46.644840 IP localhost.localdomain.47651 > localhost.localdomain.domain: 60449+ [1au] PTR? 133.5.168.192.in-addr.arpa. (67)
23:15:46.645177 IP localhost.localdomain.domain > localhost.localdomain.47651: 60449 NXDomain* 0/0/1 (55)
23:15:46.697018 IP localhost.localdomain.60023 > localhost.localdomain.domain: 2466+ [1au] PTR? 200.5.168.192.in-addr.arpa. (67)
23:15:46.697375 IP localhost.localdomain.domain > localhost.localdomain.60023: 2466 NXDomain* 0/0/1 (55)
23:15:46.750343 IP localhost.localdomain.38303 > localhost.localdomain.domain: 45756+ [1au] PTR? 88.5.168.192.in-addr.arpa. (66)
23:15:46.750702 IP localhost.localdomain.domain > localhost.localdomain.38303: 45756 NXDomain* 0/0/1 (54)
23:15:46.801254 IP localhost.localdomain.47265 > localhost.localdomain.domain: 38062+ [1au] PTR? 170.5.168.192.in-addr.arpa. (67)
23:15:46.801585 IP localhost.localdomain.domain > localhost.localdomain.47265: 38062 NXDomain* 0/0/1 (55)
23:15:46.853385 IP localhost.localdomain.37977 > localhost.localdomain.domain: 19770+ [1au] PTR? 172.5.168.192.in-addr.arpa. (67)
23:15:46.853721 IP localhost.localdomain.domain > localhost.localdomain.37977: 19770 NXDomain* 0/0/1 (55)

Extract of /var/log/pihole/pihole.log:

Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 167.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: DHCP 192.168.5.167 is mrc-lenovo-debian.mrc.network
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 96.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: DHCP 192.168.5.96 is emonTXsolar.mrc.network
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 2.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: config 192.168.5.2 is NXDOMAIN
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 139.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: config 192.168.5.139 is NXDOMAIN
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 98.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: config 192.168.5.98 is NXDOMAIN
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 95.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: config 192.168.5.95 is NXDOMAIN
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 228.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: DHCP 192.168.5.228 is robotic_cleaner.mrc.network
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 231.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: config 192.168.5.231 is NXDOMAIN
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 161.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: config 192.168.5.161 is NXDOMAIN
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 178.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: config 192.168.5.178 is NXDOMAIN
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 109.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: config 192.168.5.109 is NXDOMAIN
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 50.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: DHCP 192.168.5.50 is shellyEM.mrc.network
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 234.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: DHCP 192.168.5.234 is Tab-S7-FE-di-Marco.mrc.network
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 210.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: DHCP 192.168.5.210 is shelly1-C45BBE75C0CB.mrc.network
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 58.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: DHCP 192.168.5.58 is shelly_apertura_tapp.mrc.network
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 241.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: DHCP 192.168.5.241 is ESP-C00FFA.mrc.network
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 115.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: config 192.168.5.115 is NXDOMAIN
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 163.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: DHCP 192.168.5.163 is Chromecast.mrc.network
Sep  2 21:59:00 dnsmasq[12137]: query[PTR] 247.5.168.192.in-addr.arpa from 127.0.0.1
Sep  2 21:59:00 dnsmasq[12137]: config 192.168.5.247 is NXDOMAIN

Sudden change in queries

Query types

Top permitted domains:

Bucking_Horn · September 3, 2024, 11:13am

It doesn't seem that you've configured a DNS loop, as Pi-hole is answering those requests directly from config or DHCP, i.e. Pi-hole is not sending them upstream. If it would be a loop, you'd see Pi-hole forward request to an upstream and immediately receive an identical request from that upstream.

Pi-hole would send out reverse lookups for observed client IPs once per sharp hour, and those would go to configured upstreams, i.e. Pi-hole is not sending those requests.

That would indicate that those lookups would originate from other processes on your Pi-hole machine, and your debug log indeed shows that your Pi-hole machine is hosting a wealth of other services.

You should start looking if any of those would have reason to reverse lookup all IPs in your network, e.g. monitoring software may do so.
If there would be no obvious candidates, you could attempt to identify the offending process by disabling services and reenable them one by one.

mrc989 · September 3, 2024, 7:42pm

Thank you so much for your suggestions as always!

It took less than I expected - I started killing process after process and it soon turned out it was a Docker container, specifically Pi Alert (jokobsk/pi.alert:latest).

I killed the Docker container in the middle of a PTR flood and the queries stopped immediately.

I had installed this container about a year ago and never really used it since. No idea why, but a couple of days ago it started sending out local PTR queries every other minute. I may investigate further when I have the time.

system · September 24, 2024, 7:43pm

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.