Pi-Hole "DNS service not running" eventually

Please follow the below template, it will help us to help you!

Expected Behaviour:

The DNS service should stay up.

Actual Behaviour:

I run pi-hole and all is well for a while until it eventually stops serving requests. The status page ends up saying:

DNS service not running FTL offline
Load: 0.11 0.04 0.01
Memory usage: 9.5 %

Debug Token:

I tried to run pihole -d and run the web interface version, but it yields:

[✗] There was an error uploading your debug log.

  • Please try again or contact the Pi-hole team for assistance.
  • A local copy of the debug log can be found at: /var/log/pihole_debug.log

Then if I restart DNS I can upload this:

https://tricorder.pi-hole.net/lcelzt6dd1

run pihole -d while it is down.
do not upload the log yet
once resolution is avail again use the following command to upload the debug log from the failed state
cat /var/log/pihole_debug.log | pihole tricorder

The log outputs in your debug log show the problem. You have a huge number of DNS requests, and they eventually overwhelm the capabilities of your Pi (appears to be a Zero?).

   [2020-01-11 05:38:16.147 19788] Resizing "/FTL-queries" from 140967936 to 141164544
   [2020-01-11 05:38:16.194 19788] Imported 2938661 queries from the long-term database
   [2020-01-11 05:38:16.198 19788]  -> Total DNS queries: 2938661
   [2020-01-11 05:38:16.199 19788]  -> Cached DNS queries: 10759
   [2020-01-11 05:38:16.199 19788]  -> Forwarded DNS queries: 2926812
   [2020-01-11 05:38:16.200 19788]  -> Exactly blocked DNS queries: 1090
   [2020-01-11 05:38:16.200 19788]  -> Unknown DNS queries: 0
   [2020-01-11 05:38:16.200 19788]  -> Unique domains: 498
   [2020-01-11 05:38:16.200 19788]  -> Unique clients: 25
   [2020-01-11 05:38:16.201 19788]  -> Known forward destinations: 3

You have almost 3 million DNS requests in 24 hours, which is likely circular traffic due to conditional forwarding. Please post the outputs of the following commands from the Pi terminal:

echo ">top-clients" | nc localhost 4711

echo ">top-domains" | nc localhost 4711

tail -n25 /var/log/pihole.log

Thanks for the response. This might be due to what I’m trying to do on my network - please let me know if it’s crazy and there’s a better way to do it.

I have a Ubiquiti USG (10.0.0.2) that handles IP assignments since it’s got a nice UI for static IPs and the like. I figured I’d have it point it’s clients to my pi, (10.0.0.3) and all would be merry.

Here’s the information you asked for:

root@pihole:/etc/dnsmasq.d# echo ">top-clients" | nc localhost 4711

tail -n25 /var/log/pihole.log0 2254 10.0.20.2
1 1480 10.0.20.3
2 38 127.0.0.1 localhost
3 6 10.0.0.2 unifi
4 2 10.0.110.6
5 2 10.0.0.157
6 2 10.0.0.90 optimus.home.mydomain.com
7 2 10.0.111.6
8 1 10.0.0.190
9 1 10.0.0.165
---EOM---



root@pihole:/etc/dnsmasq.d# echo ">top-domains" | nc localhost 4711
0 2752 panda-rancher-01.home.mydomain.com
1 2742 panda-rancher-01.home.mydomain.com.home.mydomain.com
2 22 panda-rancher-03.home.mydomain.com
3 10 panda-rancher-02.home.mydomain.com
4 8 www.gstatic.com
5 6 rancher.home.mydomain.com
6 6 s3.amazonaws.com
7 6 api.github.com
8 4 ping.ui.com
9 3 3.20.0.10.in-addr.arpa
---EOM---


root@pihole:/etc/dnsmasq.d# tail -n25 /var/log/pihole.log
Jan 14 01:32:44 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com to 10.0.0.1
Jan 14 01:32:44 dnsmasq[15262]: query[A] panda-rancher-01.home.mydomain.com from 10.0.20.2
Jan 14 01:32:44 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com to 10.0.0.1
Jan 14 01:32:44 dnsmasq[15262]: query[AAAA] panda-rancher-01.home.mydomain.com from 10.0.20.2
Jan 14 01:32:44 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com to 10.0.0.1
Jan 14 01:32:44 dnsmasq[15262]: query[AAAA] panda-rancher-01.home.mydomain.com.home.mydomain.com from 10.0.
20.3
Jan 14 01:32:44 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com.home.mydomain.com to 10.0.0.1
Jan 14 01:32:44 dnsmasq[15262]: query[A] panda-rancher-01.home.mydomain.com.home.mydomain.com from 10.0.20.
3
Jan 14 01:32:44 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com.home.mydomain.com to 10.0.0.1
Jan 14 01:32:44 dnsmasq[15262]: query[A] panda-rancher-01.home.mydomain.com.home.mydomain.com from 10.0.20.
2
Jan 14 01:32:44 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com.home.mydomain.com to 10.0.0.1
Jan 14 01:32:44 dnsmasq[15262]: query[AAAA] panda-rancher-01.home.mydomain.com.home.mydomain.com from 10.0.
20.2
Jan 14 01:32:44 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com.home.mydomain.com to 10.0.0.1
Jan 14 01:32:44 dnsmasq[15262]: query[A] panda-rancher-01.home.mydomain.com from 10.0.20.2
Jan 14 01:32:44 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com to 10.0.0.1
Jan 14 01:32:44 dnsmasq[15262]: query[AAAA] panda-rancher-01.home.mydomain.com from 10.0.20.2
Jan 14 01:32:44 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com to 10.0.0.1
Jan 14 01:32:44 dnsmasq[15262]: query[AAAA] panda-rancher-01.home.mydomain.com.home.mydomain.com from 10.0.
20.2
Jan 14 01:32:44 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com.home.mydomain.com to 10.0.0.1
Jan 14 01:32:44 dnsmasq[15262]: query[A] panda-rancher-01.home.mydomain.com.home.mydomain.com from 10.0.20.
2
Jan 14 01:32:44 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com.home.mydomain.com to 10.0.0.1
Jan 14 01:32:44 dnsmasq[15262]: query[AAAA] panda-rancher-01.home.mydomain.com from 10.0.20.3
Jan 14 01:32:45 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com to 10.0.0.1
Jan 14 01:32:45 dnsmasq[15262]: query[A] panda-rancher-01.home.mydomain.com from 10.0.20.3
Jan 14 01:32:45 dnsmasq[15262]: forwarded panda-rancher-01.home.mydomain.com to 10.0.0.1
root@pihole:/etc/dnsmasq.d#

My dnsmsq.d has a 02-custom.conf that might be of interest as well:

root@pihole:/etc/dnsmasq.d# cat 02-custom.conf
# Could use a LB in front of it?
address=/internal.home.mydomain.com/10.0.110.3
# Could use a LB in front of it?
address=/external.home.mydomain.com/10.0.111.3
address=/rancher.home.mydomain.com/10.0.10.0
address=/pihole.home.mydomain.com/10.0.0.3

# metallb assigned
address=/unitycache.home.mydomain.com/10.0.110.202

domain=home.mydomain.com,10.0.0.0/16,local

Please generate a new debug log and post the token. The old one is expired and the outputs you show don’t match the traffic volume reported in the earlier debug log.

New debug token: https://tricorder.pi-hole.net/y9ybdzki7k. I’m assuming this has the logs you need?

I should mention that in the intervening time I noticed that one of the hosts it was looking for repeatedly was down. I since rebooted that machine and now my dns traffic seems to have stabilized. I’m guessing the error is still there, but it only shows itself when a host is down?

The volume of traffic appears to have decreased significantly, to 325K requests in the last 24 hours. You may have resolved the problem.

Oh; so you think the issue was just a set of clients being naughty? I was wondering if I somehow had pihole stuck in a loop when this error condition happened.

Tale a look at your query volume in 12 hours. The data shown is a rolling 24 hour average and you want to look 24 hours after you fixed that client.