Lose connectivity to my network after a certain number of cached DNS entries

[jaquilina]

Good Morning Everyone,

I have a strange issue where after a certain number of cached DNS entries my network goes down sometimes it comes back up on its own other times i need to restart the DNS resolver.

I also see coming up warnings as follows:

Ignoring query from non-local network

I have my LAN setup to use the pihole for resolution and then my security appliance. So if the pihole does not find what it needs it forwards the query to my security appliance.

Prior to this happening i had a network cable running to this part of my LAN but have had to come up with a temporary solution as the cable needs to be replaced. I am now using Home plugs.

I am not sure what is going on or if this is even a bug with pihole itself.

[jfb]

Please upload a debug log and post just the token URL that is generated after the log is uploaded by running the following command from the Pi-hole host terminal:

pihole -d

or do it through the Web interface:

Tools > Generate Debug Log

[jaquilina]

Hi Jfb,

URL is https://tricorder.pi-hole.net/NfwvLK59/

[yubiuser]

Your debug log expired. Please upload a new one.

[jaquilina]

here is a new debug post: https://tricorder.pi-hole.net/iM3mJHx1/

I want to add i did increase the cache size and what i did see is that all weekend the connectivity was fine until this morning.

[yubiuser]

I don't see any error in your debug log that would explain what you are experience.

Why do you suspect that this is due to insufficient cache size?

What is the output of

echo ">stats >quit" | nc 127.0.0.1 4711
echo ">cacheinfo >quit" | nc 127.0.0.1 4711

[jaquilina]

Hey Yubi,

I am suspecting that the DNS cache is not removing old records and I would have over 10,000 cached records. When I flush the DNS cache things seem to return to normal. That is also the reason I bumped up the cache size to 100k

domains_being_blocked 100077
dns_queries_today 29199
ads_blocked_today 1688
ads_percentage_today 5.781020
unique_domains 3174
queries_forwarded 19476
queries_cached 6748
clients_ever_seen 33
unique_clients 25
dns_queries_all_types 29199
reply_NODATA 2045
reply_NXDOMAIN 1010
reply_CNAME 16987
reply_IP 7369
privacy_level 1
status enabled

cache-size: 100000
cache-live-freed: 0
cache-inserted: 76193
ipv4: 126
ipv6: 21
srv: 0
cname: 358
ds: 0
dnskey: 0
other: 26
expired: 1197
immortal: 40

[yubiuser]

@jaquilina

cache-live-freed: 0

Not a single cache entry needed to be removed although it hadn't expired so far.

What is the optimal cache size?

Cache sizes of more than maybe 1000 are only useful in very specific environments (very heterogeneous clients querying a large amount of different domains). I added a cache efficiency measure into FTLDNS (similar to what dnsmasq offers). Although it is not (yet) exposed nicely on the GUI, you can query it manually. Run echo ">cacheinfo" | nc pi.hole 4711 on your Pi-hole. You should get something like cache-size: 1000 cache-live-freed: 0 cache-inserted: 12620 The individual numbers mean the…

To find out if the cache is really the issue here, please decrease it again to the default 10000 and wait until you experience the issue. Then run the command again and check the cache-live-freed

[jaquilina]

Hi Yubi,

I have isolated the issue to another part of my network. I have a question though im seeing a lot of DNSMASQ warnings. In particular the warning message is ignoring query from non local network. Can you advise if there is a way I can see what non local network is trying to query my network?

Regards,

Jonathan

[yubiuser]

@jaquilina

I can see what non local network is trying to query my network?

The next version of Pi-hole will contain the requesting IP. This version is already at the horizon....

[jaquilina]

I have had the issue again I am 10,000 set as the cache size but i have over 300,000 records in the cache live freed command where do i run that command from?

[jfb]

Please post a new output of the following command from the Pi terminal:

echo ">cacheinfo >quit" | nc 127.0.0.1 4711

[jaquilina]

cache-size: 10000
cache-live-freed: 0
cache-inserted: 2488
ipv4: 74
ipv6: 9
srv: 0
cname: 145
ds: 0
dnskey: 0
other: 23
expired: 507
immortal: 40

The cache live freed since that is 0 does that mean that old records are not being removed?

[jfb]

@jaquilina

cache live freed since that is 0 does that mean that old records are not being removed?

Cache entries are not being forcibly removed prior to the end of their TTL in order to make space for new entries. Cache entries are expiring from the cache naturally at the end of the TTL.

@jaquilina

cache-size: 100000
cache-live-freed: 0
cache-inserted: 76193

In your original output, none of the entries in the cache were forcibly evicted. The data shown is cumulative from the last time that FTL was restarted. In that time, you had 76193 entries into the cache. But, even with a cache size of 10000, it is highly unlikely (given your network query traffic volume) that any of these would be forcibly evicted from the cache.

It is quite normal to have the number of cache insertions much greater than the cache size, depending on how long FTL has been running. The parameter that tells you if the cache is overflowing is cache-live-freed. If that is zero, you don't need a larger cache (nor do you want one, as this consumes memory unnecessarily).

This is similar in concept to putting gas into your car's tank. The tank may hold 10 gallons, and in a year you may consume 500 gals, but the tank never overflows. Gas is consumed faster than you put it in.

[jfb]

@jaquilina

Can you advise if there is a way I can see what non local network is trying to query my network?

This functionality is in our latest release that was issued today.

Listed in Commits on Jan 12, 2022 - Log source of ignored query when local-service is used

Comparing v5.13...v5.14 · pi-hole/FTL

[jaquilina]

Morning jfb,

Thanks for your reply. The strange thing is I have very random hiccups which seem to disappear upon restarting the DNS resolver.

The funny thing its only the part of my network which is currently operating over a home plug until I can run a new network cable. Could this randomness be caused by dnsmasq in some way? Are there any other logs I can check to see any potential errors?

[jfb]

@jaquilina

Could this randomness be caused by dnsmasq in some way?

I don't think so in this case. Your common problem area in the network is the home plug, and that is likely the root of the problem.

All the Pi-hole log entries are made in /var/log/pihole-FTL.log and /var/log/pihole.log. The dnsmasq warnings are also carried in the diagnostics page in the web admin GUI.

[jaquilina]

I have updated ot the latest version. I will have to wait until it happens again and those warning surface again.

Will have to provide feedback on this thread once it happens again.

[jaquilina]

Had another network hiccup now and it seems like a different segment of my lan is causing these to be generated:

ignoring query from non-local network 192.168.113.2 (logged only once), did a bit of digging on this ip and this is the ip address given out by my vpn to my machine. What about this is making DNSMASQ unhappy?

I am not sure why this is starting to be logged now and these hiccups are occuring. When these hiccups occur i cannot load any website. This happened to me and im outside the home network connected to vpn.

[yubiuser]

Please upload a debug log and post just the token URL that is generated after the log is uploaded by running the following command from the Pi-hole host terminal:

pihole -d

or do it through the Web interface:

Tools > Generate Debug Log

---

@jaquilina

I am not sure why this is starting to be logged now and these hiccups are occuring.

It has been logged always, but only to the log file. Recently, we made them appear in the web interface as well. This is when users started to notice them.