I am running Pi-hole in a Docker container for a few devices on my network. These devices are configured with a DHCP reservation which sets their DNS servers to my Pi-hole instance, rather than the router itself which is the resolver for other network clients.
Despite not being configured to use Pi-hole for DNS, my router is sending thousands of requests to Pi-hole which results in it being constantly rate-limited.
I've disabled rate limiting for the time being, but I'd like to know what the correct fix is. (edit: I've had to switch rate limiting back on since the web interface becomes unusable with the high amount of queries. I've set it to 100/10 for the time being)
Below is a screenshot showing this behaviour, thousands of queries within the same second from my router (gateway.myinternal.domain) to one of my servers (portainer.myinternal.domain)
I've googled around but all the results I can find for the rate-limiting issue are from clients actually configured to use Pi-hole.
Pi-Hole is set up to use my router as its upstream DNS server so I can still resolve all my internal domain names (I've got this all set up in pfSense and don't want to use Pi-Hole's 'Local DNS' feature for this). My router is then configured to use Quad9 for its upstream DNS.
Pi-hole reports what it sees. There may be some circular logic going on, but a debug log may help. pihole -d or Tools > Generate debug log and post the token URL provided.
I had a look through the debug log and couldn't see any relevant information, and I'd rather not upload it since it contains information about my network and the domain names I use.
I've worked around the issue by adding a local firewall rule on the box that runs Pi-hole to drop any incoming traffic on 53/UDP from my router since there will never be a situation on my network where this traffic is neccesary.
Only a few moderators/developers (less than 10 people) will have access to your debug log when you choose yes to upload the log on the command line (or select the checkbox on the web interface).
Your log will be uploaded to a secure server. No one else will be able to read it.
Ah, thank you, I wasn't aware of this - I've managed to resolve this using a firewall rule but I'll keep this in mind if I encounter any issues in future.
Your DNS storm is caused by one or some of your 'other network clients' wanting to resolve the ip address of portainer.whatever. If you added an entry to those 'other network clients' host file they wouldn't be constantly making the dns requests. You could try adding a DNS record (Local DNS - DNS Records) on your pihole.
What I do not undestand is you say 'Pi-Hole is set up to use my router as its upstream DNS server' yet your first image shows your router is a client of your pihole.
That's the confusing thing, there are currently only two devices on my network using Pi-hole for DNS resolution - my MacBook and my iPhone. Nothing else should be talking to Pi-Hole for DNS resolution. Any other machines wanting to resolve portainer.myinternal.domain will use my router for DNS, just as they did before I implemented Pi-Hole.
I can absolutely garauntee that my router is not configured in any way to use Pi-hole as its upstream DNS server. It's configured only to use Quad9 with cloudflare as a backup. This has been proved by the fact that after blocking UDP/53 from my router to the Pi-hole instance, this activity has ceased with no impact to my network.
Humour me. Your router is running pfsense? Go to Diagnostics/DNS Lookup and lookup your portainer. Confirm that below Timings, the DNS server queried is not your pihole.
When the firewall rule blocking queries from the router alleviated the issue it confirms that the router was causing the problem - for certain queries a dns loop existed. Within pfsense have you configured Domain Overrides?
You, sir, are a gentleman and a scholar - I think I've cracked it.
I had a domain override configured for myinternal.domain pointing to the server all my Docker containers run on - I didn't realise this actually forwarded DNS queries, I thought it just acted as a DNS record (why spend a few minutes reading the documentation when you can spend a few hours troubleshooting?)
Since I never had a DNS resolver running on this IP until now I guess it was just sending requests into the void. This also explains some weird issues with long load times for my internal domains, I guess pfSense way trying to talk to a non-existant DNS server for a while before timing out and falling back to the manual DNS entries I have configured.
(why spend a few minutes reading the documentation when you can spend a few hours troubleshooting?)