DNS stopped working after update

Expected Behaviour:

DNS service should be running.

Actual Behaviour:

DNS service is not running

$ pihole status
  [✗] DNS service is NOT running

I updated to latest version yesterday. DNS worked all day. But this morning it has stopped working.When I restart DNS with pihole restartdns, the service starts for a few seconds and then dies.

Debug Token:

99ty09ucy6

Thanks!

This line in your pihole-FTL log indicates that there is very heavy traffic volume to the Pi.

[2020-01-25 09:43:22.383 8124] Resizing "/FTL-queries" from 319881216 to 320077824

What is the output of the following commands from the Pi terminal - you can copy/paste the text output in a reply. We're looking to see what's causing all the traffic, this is likely what is keeping FTL from starting.

echo ">stats" | nc localhost 4711

echo ">top-clients withzero (15)" | nc localhost 4711

echo ">top-domains" | nc localhost 4711

echo ">top-ads" | nc localhost 4711

Thanks. I am pasting output of commands below. I did have to run pihole restartdns to be able to run the commands. DNS stopped shortly afterwards.

$ echo ">stats" | nc localhost 4711
domains_being_blocked 77227
dns_queries_today 10122120
ads_blocked_today 3421
ads_percentage_today 0.033797
unique_domains 2290
queries_forwarded 10117278
queries_cached 1421
clients_ever_seen 24
unique_clients 24
dns_queries_all_types 10122120
reply_NODATA 0
reply_NXDOMAIN 0
reply_CNAME 25
reply_IP 15
privacy_level 0
status enabled
---EOM---
echo ">top-clients withzero (15)" | nc localhost 4711
0 10101502 10.10.0.1 usg-3p
1 9134 10.10.0.187 saties-imac.<REDACT>
2 4678 10.10.0.188 macd1512ssharma.<REDACT>
3 2383 10.10.0.189 iphone.<REDACT>
4 1383 10.10.0.25 ipad-2.<REDACT>
5 728 127.0.0.1 localhost
6 440 10.10.40.11 
7 394 10.10.0.81 saties-ipad-pro.<REDACT>
8 378 10.10.0.85 living-room.<REDACT>
9 373 10.10.40.19 
10 237 10.10.0.105 
11 104 10.10.0.97 ishitas-air.<REDACT>
12 89 10.10.0.112 unifi
13 75 10.10.0.104 switch8-60w.<REDACT>
14 73 10.10.40.18 
---EOM---
echo ">top-domains" | nc localhost 4711
0 2454814 lb._dns-sd._udp.<REDACT>
1 1866727 b._dns-sd._udp.<REDACT>
2 1863291 db._dns-sd._udp.<REDACT>
3 1622075 lb._dns-sd._udp.0.0.10.10.in-addr.arpa
4 784163 db._dns-sd._udp.0.0.10.10.in-addr.arpa
5 783288 b._dns-sd._udp.0.0.10.10.in-addr.arpa
6 145082 _aaplcache1._tcp.<REDACT>
7 145080 _aaplcache3._tcp.<REDACT>
8 145080 _aaplcache4._tcp.<REDACT>
9 145079 _aaplcache2._tcp.<REDACT>
---EOM---
echo ">top-ads" | nc localhost 4711
0 751 mobile.pipe.aria.microsoft.com
1 233 device-metrics-us-2.amazon.com
2 180 graph.instagram.com
3 170 nexus.officeapps.live.com
4 161 www.googleadservices.com
5 125 reports.crashlytics.com
6 106 www.google-analytics.com
7 92 www.googletagmanager.com
8 88 googleads.g.doubleclick.net
9 50 js-agent.newrelic.com
---EOM---

home.satie.io is accessible on the internet as a valid domain. Using that same domain locally will cause issues if you do not have a resolver set to answer queries for that domain.

You have conditional forwarding enabled. Pi-hole is forwarding queries for that domain to the USG for home.satie.io and USG is asking Pi-hole for information about that domain. Note that usg-3p is the top client for Pi-hole. It should not be a client if it's set as an upstream or a conditional forwarding server.

EDIT: This isn't accurate, it seems that the parent domain satie.io is at Cloudflare?

Original:

Note, the TTL of 10 seconds is very, very low and will cause an extremely high volume of queries.

dschaper@Mariner-10:~$ dig home.satie.io

; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> home.satie.io
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17751
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;home.satie.io.                 IN      A

;; ANSWER SECTION:
home.satie.io.          10      IN      A       <REDACT>
home.satie.io.          10      IN      A       <REDACT>

;; Query time: 1 msec
;; SERVER: 192.168.88.1#53(192.168.88.1)
;; WHEN: Sat Jan 25 10:23:47 PST 2020
;; MSG SIZE  rcvd: 63

Thanks. It is weird that you can resolve home.satie.io. I use this subdomain for the home network exclusively, and have no public DNS entries anywhere.

It seems like the USG may be the culprit here. I see the preferred DNS settings seem to have changed overnight to 127.0.0.1. I will force a provision (settings update) to set the DNS to pihole's IP and restart pihole.

pi@noads:~ $ host <REDACTED>
Host <REDACTED> not found: 3(NXDOMAIN)

pi@noads:~ $ whois <REDACTED>
NOT FOUND
1 Like

However, the top domain satie.io seems to taken, as a respective nslookup reveals:

Addresses:  
       2606:4700:3034::681c:1f93
       2606:4700:3035::681c:1e93
       104.28.31.147
       104.28.30.147
1 Like
pi@noads:~ $ whois <REDACTED>
[..]
Registrant Organization:
Registrant State/Province: <REDACTED>
Registrant Country: <REDACTED>
Name Server: <REDACTED>
Name Server: <REDACTED>
[..]
1 Like

Thanks. Yes, I have a satie.io on Cloudflare. :slight_smile:

This seems to be resolved for now. Unifi firmware updates last night may be the reason for the problem. I will monitor throughout the day.

Thanks to all for their help!

2 Likes

If feel uncomfortable, I can remove the reveals :wink:

1 Like

I just checked again and what appears to have happened is an error on my side. The home subdomain is NXDOMAIN. I turn off my local Pi-hole installs regularly to see how things are without any protection and my ISP was intercepting the NXDOMAIN response and inserting their own A records.

Using a subdomain of an active domain can be tricky though. Normally Pi-hole will not forward any requests for private domains or private address ranges. When you use TLDs that can be resolved on the internet with public DNS servers you lose that protection.

So in this case it looks like a loop between Pi-hole and USG. I suspect that one of the logs has a number of "Max queries (150) exceeded" warnings before things crash out.

1 Like

As a troubleshooting step, I would disable conditional forwarding. Conditional forwarding has been known to cause significant DNS looping. It's an easy thing to turn it off and on, so turn it off and see if the traffic volume drops to normal.

1 Like

Sure. Thanks.

Thanks. Will remove conditional forwarding and test.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.