DNS resolution breaks when temporarily disabling Pi-Hole 5

Please follow the below template, it will help us to help you!

Expected Behaviour:

Disabling Pi-Hole for the defined period of time allows blocked elements through while maintaining DNS resolution

Actual Behaviour:

DNS resolution breaks network-wide for the duration of the selected time until the disabled period is over.

Debug Token:

https://tricorder.pi-hole.net/vt4i23osx4

seems like a similar issue to https://www.reddit.com/r/pihole/comments/gvbgwq/dns_dies_when_i_tell_it_pihole_to_pause/, but in my case, resolution restarts after a period of time.

After you disable Pi-hole, what are the outputs of these commands:

From the Pi:

nslookup pi.hole

nslookup flurry.com

From a client:

nslookup pi.hole

nslookup flurry.com

from pi:

pi@pi-hole:~ $ nslookup pi.hole
;; connection timed out; no servers could be reached

pi@pi-hole:~ $ nslookup flurry.com
;; connection timed out; no servers could be reached

from client:

nslookup pi.hole
DNS request timed out.
    timeout was 2 seconds.
Server:  UnKnown
Address:  192.168.1.13

DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to UnKnown timed-out
nslookup flurry.com
DNS request timed out.
    timeout was 2 seconds.
Server:  UnKnown
Address:  192.168.1.13

DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to UnKnown timed-out

This is still occurring. The problem is solved by a reboot temporarily. Any ideas?

1 Like

If that outage occurs, what's the output when running the following commands on your Pi-hole machine:

pihole status
sudo systemctl status --full --no-pager pihole-FTL.service

Would a restart of Pi-hole fix your issue?

pihole restartdns

What's the output of the same above systemctl command right after this restart?

I'm seeing the same issue, here are some more details about the behavior I'm seeing, and the timeline of steps I'm following. (Times are MM:SS)

00:00 - Using the admin console ("Disable" on the side menu,) I disable Pi-Hole for 60 seconds.
00:00 through 01:00 - I make requests from different devices on my network. Any requests that do not have a cached DNS entry fail. pihole status returns the following:

  [✓] DNS service is running
  [✗] Pi-hole blocking is Disabled

01:00 through 06:13 - All non-cached DNS requests continue to time out.

Running pihole status during this time yields the following:

  [✓] DNS service is running
  [✓] Pi-hole blocking is Enabled

If I run pihole restartdns it fixes DNS resolution.

I'm wondering if the 6 minute and 13 second dead window breaks down as follows:

  • 1 minute of dead time that was requested (though, this shouldn't break DNS resolution all together, right?)
  • 5 minutes for some cache entry to expire?
  • 13 seconds for the DNS service to start up or restart after that cache entry expires.

Couple of questions in summary:

  • Disabling pihole through the admin console shouldn't break DNS resolution all together, right? (I have my pihole setup as my DHCP server, and have my router pointing to my local pihole address for primary DNS resolution. My router has an input for a secondary DNS server, but I have 0.0.0.0 in there. Wondering if this may be why DNS resolution breaks during the disabled period. Any insight?)

Bucking_Horn, I'm not OP, but here is my output to the commands you're suggesting. When I ran the first command, DNS resolution was broken. But it works again after the completion of restartdns.

pi@raspberrypi:~ $ sudo systemctl status --full --no-pager pihole-FTL.service
● pihole-FTL.service - LSB: pihole-FTL daemon
   Loaded: loaded (/etc/init.d/pihole-FTL; generated)
   Active: active (exited) since Thu 2020-06-25 09:29:41 EDT; 33min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 25935 ExecStart=/etc/init.d/pihole-FTL start (code=exited, status=0/SUCCESS)

Jun 25 09:29:36 raspberrypi systemd[1]: Starting LSB: pihole-FTL daemon...
Jun 25 09:29:36 raspberrypi pihole-FTL[25935]: Not running
Jun 25 09:29:36 raspberrypi su[25957]: (to pihole) root on none
Jun 25 09:29:36 raspberrypi su[25957]: pam_unix(su:session): session opened for user pihole by (uid=0)
Jun 25 09:29:41 raspberrypi pihole-FTL[25935]: FTL started!
Jun 25 09:29:41 raspberrypi systemd[1]: Started LSB: pihole-FTL daemon.
pi@raspberrypi:~ $ pihole restartdns
  [✓] Restarting DNS server
pi@raspberrypi:~ $ sudo systemctl status --full --no-pager pihole-FTL.service
● pihole-FTL.service - LSB: pihole-FTL daemon
   Loaded: loaded (/etc/init.d/pihole-FTL; generated)
   Active: active (exited) since Thu 2020-06-25 10:03:00 EDT; 2s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 30207 ExecStart=/etc/init.d/pihole-FTL start (code=exited, status=0/SUCCESS)

Jun 25 10:02:55 raspberrypi systemd[1]: Starting LSB: pihole-FTL daemon...
Jun 25 10:02:55 raspberrypi pihole-FTL[30207]: Not running
Jun 25 10:02:55 raspberrypi su[30231]: (to pihole) root on none
Jun 25 10:02:55 raspberrypi su[30231]: pam_unix(su:session): session opened for user pihole by (uid=0)
Jun 25 10:03:00 raspberrypi pihole-FTL[30207]: FTL started!
Jun 25 10:03:00 raspberrypi systemd[1]: Started LSB: pihole-FTL daemon.

Any help is appreciated!

Bull, am I correct that your observed behaviour differs slightly from OP, as you have to actively restart Pi-hole where OP returns back to operational once the disable period expires?

For completeness, it would be good to know if you can confirm your output for the nslookups as requested by jfb would match the results as observed by g_p.

Your output looks perfectly normal - it doesn't provide any hints as to why Pi-hole seems to stop DNS resolution completely instead of just stop DNS filtering.

As pihole-FTL is reported as running, nslookup results timing out would imply that something interferes with DNS requests getting through to pihole-FTL, or pihole-FTL rejects DNS requests actively.

Unfortunately, I am not able to reproduce your observations on my side so far.

In lieu of any more specific leads to follow
What are you using as your Pi-hole's upstream DNS servers, and what are the nameservers used by your Pi-hole machine (usually defined in /etc/resolv.conf and/or /etc/dhcpcd.conf)?
And @g_p, are you using Pi-hole as DHCP server as well?

Well this is frustrating... Originally when I found this thread, I did follow the steps mentioned by jfb and observed the same results that g_p observed. (I forgot to mention that in my previous post.) Since you explicitly asked, I figured I'd follow those steps again and post my output.

However I'm no longer able to reproduce any of these issues! When I disable pihole now, DNS resolution continues to work exactly as expected. (In other words, only DNS filtering is disabled, as it should be.)

I've noticed this issue pop up multiple times over the last few weeks (more than just yesterday.) So I'm confident that it will resurface, but since I can no longer reproduce it reliably, I'm not sure how we can proceed with debugging it.

Just to answer your questions:

Bull , am I correct that your observed behaviour differs slightly from OP, as you have to actively restart Pi-hole where OP returns back to operational once the disable period expires?

My observed behavior differs slightly, but not exactly as you indicated. For my case, actively restarting pihole does fix DNS resolution. But it isn't required. Once the disabled period ends, after an additional period of about 5 minutes DNS resolution resumes normal operation.

What are you using as your Pi-hole's upstream DNS servers, and what are the nameservers used by your Pi-hole machine (usually defined in /etc/resolv.conf and/or /etc/dhcpcd.conf )?

In my Pi-Hole settings, I have OpenDNS (ECS) configured for both IPv4 checkboxes. Nothing else in that table is checked. The table to the right, for Custom Upstream DNS Servers is completely blank.

/etc/resolv.conf contains only a single (non-comment) line: nameserver 127.0.0.1
/etc/dnscd.conf is a larger file, but it ends with this reference to name_server

# fallback to static profile on eth0
#interface eth0
#fallback static_eth0
interface eth0
        static ip_address=192.168.1.164/24
        static routers=192.168.1.1
        static domain_name_servers=127.0.0.1

(164 is the address of my PiHole.)

Thanks for taking the time to look into this with me

I have the same issue as OP.
Disable Pi-Hole
DNS breaks
DNS works again when pihole is back up.

I restarted my dns per DNS resolution breaks when temporarily disabling Pi-Hole 5 before pulling logs and the issue was resolved.

I can pull any logs needed to help diagnose.

At least, Pi-hole is again behaving as expected for you. :wink:

(That's a typo for the filename, isn't it?)

Nothing suspicious about your /etc/resolv.conf and /etc/dhcpcd.conf, nor about your choice of upstream DNS, so still no leads to follow. :thinking:

I'm seeing identical behavior here. When I disable Pihole for any length of time, DNS breaks and then starts working again on its own a few minutes later.

This is new behavior for me with 5.0. It didn't happen with 4.x.

I'm not seeing anyone listing their Pihole hardware here. I'm seeing this on a Pi Zero W. Not sure if that helps.

2 Likes

Same behaviour on a Pi 4

Bucking_Horn

( That's a typo for the filename, isn't it? )

Yes, sorry about that.

I started seeing the DNS resolution issues again, so I started playing around to try to find some kind of consistent reproducibility. Here's what I observed this time.

I disabled DNS filtering via the admin portal for 5 minutes. All DNS resolution broke for a period of between 30 and 90 seconds. After somewhere between 30 and 90 seconds, DNS resolution worked again, and only DNS filtering was disabled as expected. The interesting part is that this time, I observed DNS resolution working again while the filtering was still disabled. (In other words, we were still within that initial 5 minute period where I had disabled filtering, and DNS resolution began functioning again.) That wasn't something I had observed before. I was able to repeat this behavior at least 5 times.

Then I thought maybe it was a caching issue, so I decided to set my Pi-Hole DNS cache size to 0 and restart the DNS resolver. But as we've already discovered, restarting the DNS resolver fixes the issue. So I'm back to no longer being able to reproduce the issue at all. :facepalm:

(And to add to what others have said, I'm running my Pi-Hole on a Raspberry Pi 2 Model B)

Thanks for the thought, ShooflyPie.
However, there seems to be no obvious relation to CPU architecture.

So far, RPi models that encounter the issue cover ARM11 design (used in RPi 1 and Zero), Cortex-A7 (used in early RPi 2B (right, Bull?)), and Cortex-A72 (used in RPi 4B).

I am still unable to reproduce the issue, neither on my Zero nor on my RPi 3A+ (a Cortex-A53 design, also used in 3B and 3B+ as well as late RPi 2B v1.2).

EDIT: ShooflyPie, Zoaky, kusha, g_p : Are you using Pi-hole as DHCP server like Bull?

I'm not using Pihole as a DHCP server. DHCP is handled by my router (an Eero mesh, if it matters).

Is there anything I can run for you - any kind of report I can generate while it's happening that might help? I've tried to do the debug log thing...the log gets generated, but it fails to upload itself due to the lack of connectivity.

Yes, actually, there is something which will tell us what is going on, but it will create a potentially huge log file (which is a good thing in the end).

Please set

DEBUG_ALL=true

in /etc/pihole/pihole-FTL.conf and run pihole restartdns reload

The reload instruction is the same thing happening when you modify a list or change the blocking status so it should already trigger the delay. /var/log/pihole-FTL.log will now likely grow quickly with output. With the full output of the outage, we should be able to identify what is causing the delay so we can continue looking for the why and finally the how (to fix it).

Maybe trigger it twice just to ensure the debug option is already present before the delay begins.

The content of /var/log/pihole.log and /var/log/pihole-FTL.log will be of interest.

Just typing pihole restartdns reload wasn't enough to trigger an outage. I had to pause pihole within the Web UI, then I ran the command.

I've got two big log files now. Can I upload them securely somehow?

You could upload them via Pi-hole's tricorder and then post the token here:

cat some.log | pihole tricorder

(see also How do I debug my Pi-hole installation? for other ways)