Lost connection to API and Slow responses

delprimero · November 19, 2020, 3:07pm

Hi,

I've installed pihole on a Synology NAS (via a Docker container) and since switching over, the performance seems slower than my normal router setup. The setup I've used is to set pihole as the only DNS server in the UI of my router.

This morning, I noticed that all clients were not able to browse and after disabling and re-enabling the pihole, things started working again. When browsing to the admin dashboard, I'm getting the connection to the API lost message (something similar to that). Today, so far, DNS responses seem to be slow in that I am able to notice a clear distinction between what my older (router set to use 8.8.8.8 DNS) setup provided.

I've got a debug log at the time I last saw the API lost message and lost connectivity so hopefully that will help (available here: https://tricorder.pi-hole.net/8zzpgg8aw5) but so far, probably due to poor config, pihole is slower rather than quicker as I'd expect.

I've also had a look around at some posts on slow performance but as I've left my pihole setup at a fairly basic level, I'm hoping there's nothing that I have set that is causing the issue.

Any guidance would be appreciated as I'd like to get past this initial pain so that things work perfectly... so far, I'm loving the no ads internet experience... it's just the slow responses that needs addressing at this point.

Thanks in advance.

Bucking_Horn · November 20, 2020, 8:25pm

Performance issues are often associated with DNS loop configurations.
You may have configured such a partial DNS loop by enabling Conditional Forwarding:

*** [ DIAGNOSING ]: Setup variables
    REV_SERVER_CIDR=192.168.1.0/24
    REV_SERVER_TARGET=192.168.1.1
    REV_SERVER_DOMAIN=local

In additon, you are using local as your local domain name, which is the mDNS protocol default, so may potentially conflict with any devices using that.

Change that domain, and try if disabling CF changes your observed slow resolution times.

deHakkelaar · November 20, 2020, 9:30pm

Seems embedded dnsmasq code has got below nifty feature:

pi@ph5:~ $ sudo pihole-FTL -- --help | grep loop
    --dns-loop-detect   Detect and remove DNS forwarding loops.

What if you enable it with below:

sudo tee /etc/dnsmasq.d/99-dns-loop-detect.cfg <<< $'dns-loop-detect'

Restart:

pihole restartdns

And check if any DNS looping is logged:

grep 'query loop detected' /var/log/pihole.log

To undo:

sudo rm /etc/dnsmasq.d/99-dns-loop-detect.cfg

pihole restartdns

delprimero · November 20, 2020, 10:57pm

Thanks for this. I removed the local (optional) setting in the domain entry.

And @deHakkelaar, I followed your instructions to use that script and when checking the log, got no results when searching for dns loops.

grep 'query loop detected' /var/log/pihole.log

Doing some initial tests on my phone... it looks good so far. I'll keep testing it tomorrow and when the family wake up, it'll get it's real test.

Thanks for the help so far.

Bucking_Horn · November 21, 2020, 10:29am

dns-loop-detect is of limited use.

I'd advise caution when using that option. (click for more)

Contrary to what its name suggests, dns-loop-detect isn't just detecting a loop:
It will stop querying any upstream that fails the test, effectively disabling it.

And probably more important, the test is triggered only on change of Pi-hole's upstream servers.

If you change your router's configuration to close the loop, dns-loop-detect won't wave any flag at all.

Likewise, if you would change your router's configuration to break the loop, dnsmasq would continue to avoid a blocked upstream, until you apply (unnecessary) changes to Pi-hole's upstream server configurations.

This may throw you in a loop of configuration attempts where you end up dealing with issues you wouldn't face without using that option.

Potentially, dns-loop-detect could make it more difficult to analyse and address a DNS loop, escpecially if you aren't aware of its implications and/or yet unfamiliar with dnsmasq and DNS.

In common home networking scenarios with only two local DNS servers involved (Pi-hole and your router), it's less difficult to assess a loop based on configuration:
Using Pi-hole as your router's upstream will close a DNS loop if you
(a) configured your router as Pi-hole's upstream (full)
(b) enabled Pi-hole's Conditional Forwarding (partial)

Also, disabling CF is readily accessible via Pi-hole's UI and can quickly confirm whether it's involved.

Judging from the million DNS queries and from service discovery requests being forwarded to your CF target in your debug log, I consider (b) to be highly likely.

deHakkelaar · November 21, 2020, 9:34pm

I disagree with above ... just a little
When folks experience a DNS loop they cant figure out themselves, they usually seek external assistance anyway.
Common sense is checking the logs.
If router and Pi-hole host are freshly rebooted, which is common practice anyway when experiencing a though problem, those logs will reflect exactly whats going on:

pi@ph5:~ $ tail -F /var/log/pihole.log
[..]
Oct 22 18:17:41 dnsmasq[17301]: NOT using nameserver 10.0.0.1#53 - query loop detected

But I do agree that this test is only run at start of the pihole-FTL daemon.
Any changes made on the router side will go unnoticed if dont restart pihole-FTL.
And indeed when the loop is fixed, pihole-FTL also needs to be restarted to start using that upstream DNS server again that was previously excluded/disabled.
There are a bunch of if's, but every tool that makes things a little easier are welcome.

Some details:

github.com/pi-hole/pi-hole

Enable DNS forwarding loop detection

pi-hole:development ← deHakkelaar:patch-5

opened 05:05PM - 22 Oct 20 UTC

deHakkelaar

+2 -0

Found below by accident: ``` pi@ph5:~ $ man dnsmasq [..] --dns-loop…-detect Enable code to detect DNS forwarding loops; ie the situa‐ tion where a query sent to one of the upstream server eventually returns as a new query to the dnsmasq instance. The process works by generating TXT queries of the form <hex>.test and sending them to each upstream server. The hex is a UID which encodes the instance of dnsmasq sending the query and the upstream server to which it was sent. If the query returns to the server which sent it, then the upstream server through which it was sent is disabled and this event is logged. Each time the set of upstream servers changes, the test is re-run on all of them, in‐ cluding ones which were previously disabled. ``` Seems to work just fine if I point my router 10.0.0.1 back to Pi-hole in the WAN DNS settings: ``` pi@ph5:~ $ grep server= -R /etc/dnsmasq.* /etc/dnsmasq.d/01-pihole.conf:server=9.9.9.10 /etc/dnsmasq.d/01-pihole.conf:server=149.112.112.10 /etc/dnsmasq.d/01-pihole.conf:server=10.0.0.1 /etc/dnsmasq.d/01-pihole.conf:rev-server=10.0.0.0/24,10.0.0.2 /etc/dnsmasq.d/01-pihole.conf:server=/dehakkelaar.nl/10.0.0.2 /etc/dnsmasq.d/01-pihole.conf:server=/use-application-dns.net/ ``` ``` pi@ph5:~ $ sudo tee /etc/dnsmasq.d/99-my-settings.conf <<< $'dns-loop-detect' dns-loop-detect ``` ``` pi@ph5:~ $ pihole restartdns [✓] Restarting DNS server ``` ``` pi@ph5:~ $ tail -F /var/log/pihole.log [..] Oct 22 18:17:41 dnsmasq[17301]: using nameserver 10.0.0.2#53 for domain dehakkelaar.nl Oct 22 18:17:41 dnsmasq[17301]: using nameserver 10.0.0.2#53 for domain 0.0.10.in-addr.arpa Oct 22 18:17:41 dnsmasq[17301]: NOT using nameserver 10.0.0.1#53 - query loop detected Oct 22 18:17:41 dnsmasq[17301]: using nameserver 149.112.112.10#53 Oct 22 18:17:41 dnsmasq[17301]: using nameserver 9.9.9.10#53 ``` **By submitting this pull request, I confirm the following:** *please fill any appropriate checkboxes, e.g: [X]* - [x] I have read and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md), as well as this entire template. - [x] I have made only one major change in my proposed changes. - [ ] I have commented my proposed changes within the code. - [x] I have tested my proposed changes, and have included unit tests where possible. - [x] I am willing to help maintain this change if there are issues with it later. - [x] I give this submission freely and claim no ownership. - [x] It is compatible with the [EUPL 1.2 license](https://opensource.org/licenses/EUPL-1.1) - [ ] I have squashed any insignificant commits. ([`git rebase`](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html)) Please make sure you [Sign Off](https://github.com/pi-hole/pi-hole/wiki/How-to-signoff-your-commits.) all commits. Pi-hole enforces the [DCO](https://github.com/pi-hole/pi-hole/wiki/Contributing-to-the-project). --- **What does this PR aim to accomplish?:** *A detailed description, screenshots (if necessary), as well as links to any relevant GitHub issues* **How does this PR accomplish the above?:** *A detailed description (such as a changelog) and screenshots (if necessary) of the implemented fix* **What documentation changes (if any) are needed to support this PR?:** *A detailed list of any necessary changes* --- * You must follow the template instructions. Failure to do so will result in your pull request being closed. * Please respect that Pi-hole is developed by volunteers, who can only reply in their spare time.

delprimero · November 23, 2020, 10:22am

So I've gone further and removed CF (unticked the box in the UI) and notice the number of requests has dropped off massively!

I think things look better but I think a full day of use (tomorrow) will make sure I'm confident of that. Thanks for all the help thus far and hopefully, it's all over already.

Bucking_Horn · November 24, 2020, 7:24am

Just what you'd expect if you successfully break the partial DNS loop.
(As as side node: dns-loop-detect may not be able to detect such a partial loop.)

Glad it's working for you now.

As disabling CF worked, that would also imply that your router uses Pi-hole as its (hopefully sole) upstream DNS server.

That is a valid configuration, but it also means that you won't be able to identify indiviual clients in Pi-hole's Query Log, because it's indeed only your router that forwards DNS queries to Pi-hole on behalf of your clients.
This in turn may have prompted you to enable Conditional Forwarding, but CF cannot help you here: There is just one client - your router.

If you want to see individual clients in Pi-hole, you need to configure your router to distribute Pi-hole as local DNS server via DHCP.
If your router doesn't support that, you'd have to stick with your current solution. Or you could manually set Pi-hole as DNS server on each device in your network.

delprimero · November 25, 2020, 9:45am

I had a look at the router settings again and there no specific local DNS option. I’m likely to move DHCP away from the router and over to the Synology, to save time and effort on setup when I change internet supplier. That might help to circumvent that issue longer term.

delprimero · November 25, 2020, 9:46am

Things are clearly improved as yesterday was without issue. I think it’s sorted. Thanks for all the help here, it’s appreciated.

system · December 5, 2020, 6:17pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.