After a Pi-Hole update from a few weeks ago, my Pi-Hole keeps crashing due to RAM shortage:
2022-03-31 20:06:21
SHMEM
RAM shortage ( `/dev/shm` ) ahead: **99% used**
/dev/shm: 483.9MB used, 484.0MB total, FTL uses 483.9MB
I expect Pi-Hole not to crash. It previously worked for months without any issues.
This is telling you that you are out of shared memory:
https://docs.pi-hole.net/ftldns/configfile/#check_shmem
*** [ DIAGNOSING ]: Disk usage
Filesystem Size Used Avail Use% Mounted on
/dev/root 59G 3.9G 52G 7% /
devtmpfs 430M 0 430M 0% /dev
tmpfs 462M 462M 132K 100% /dev/shm
tmpfs 462M 24M 438M 6% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 462M 0 462M 0% /sys/fs/cgroup
/dev/mmcblk0p1 253M 50M 203M 20% /boot
How many queries has Pi-hole processed in the past 24 hours? What is the output of the following from the Pi terminal:
echo ">stats >quit" | nc localhost 4711
Output:
domains_being_blocked 120371
dns_queries_today 10941370
ads_blocked_today 373
ads_percentage_today 0.003409
unique_domains 945
queries_forwarded 10692864
queries_cached 4023
clients_ever_seen 45
unique_clients 45
dns_queries_all_types 10941370
reply_NODATA 217
reply_NXDOMAIN 7
reply_CNAME 339
reply_IP 276
privacy_level 0
status enabled
That will do it. Almost 11 million DNS queries in 24 hours. From the Pi terminal, what are the outputs of the following:
echo ">top-clients >quit" | nc localhost 4711
echo ">top-domains >quit" | nc localhost 4711
Outputs:
echo ">top-clients >quit" | nc localhost 4711
0 10928875 10.0.0.1
1 1652 10.0.3.3 david-iphone.sanchez.is
2 1546 10.0.4.57 Davids-Air.sanchez.is
3 1369 10.0.0.20 nas-1.sanchez.is
4 1085 10.0.0.11 mac-server.sanchez.is
5 992 10.0.1.3 edgar-iphone.sanchez.is
6 895 10.0.0.40 living-room-apple-tv.sanchez.is
7 792 10.0.0.22 nas-3.sanchez.is
8 596 10.0.3.4 david-ipad.sanchez.is
9 593 10.0.0.42 gym-apple-tv.sanchez.is
echo ">top-domains >quit" | nc localhost 4711
0 10926975 lb._dns-sd._udp.sanchez.is
1 1052 edgariscoding.unmsapp.com
2 878 api.insight.synology.com.sanchez.is
3 774 api.insight.synology.com
4 676 gateway.fe.apple-dns.net
5 475 manifest-dp.hulu.com.akadns.net
6 388 google-public-dns-a.google.com
7 386 smtp.office365.com
8 290 spvsr.wtng.io
9 266 clientsapigw.us-east-1.gws.ring.com
This is the root cause of your problem. This client (the router?) is making a huge number of queries and exhausiting available memory.
Thank you, I found a discussion related to this issue on Reddit, I believe you commented there as well. I'll research this and try to get it resolved, the thing I'm confused about is why it just now started being an issue. Pi-Hole hadn't crashed before.
Thanks for your help!
You likely did not have this query volume in the past.
I also run Apple devices (many) on my network and in 24 hours I get maybe a few hundred of this type of query in a day. Across my 5 Pi-holes there a total of 132 in the past 24 hours.
Nothing has changed in my network though, no new devices, no guest network, and no new users, and it's a private network. I guess that doesn't mean anything, something could have changed on the existing devices themselves via software/firmware updates, right?
Also, my Pi has 1GB of physical RAM. /dev/shm is 462MB, is this something I could increase to 1GB? Sorry if it seems like a basic question.
I am also having an issue with two Pi Holes crashing due to over a million DNS queries from my router. Why would my router just now be overwhelming my PiHole with internal queries? Conditional formatting? I'll try disabling that.
It's definitely due to conditional formatting, I turned it off and mine hasn't crashed since.
Now the question is what happened that is causing it to crash now, considering there's been zero changes to my network config, no new devices, etc. I've definitely updated pi-hole any time I see a new update available so maybe there's some changes that broke this functionality.
I think you mean Conditional Forwarding here. 
Enabling CF may close a partial DNS loop if your router is configured to use Pi-hole as its upstream DNS server at the same time.
Configuring your router in that way may be required if your router does not allow to distribute Pi-hole as local DNS server via DHCP (but in such a configuration, trying to associate clinet IPs with hostnames by enabling CF would be useless anyway, as all DNS requests would originate from your router).
In your case, it would seem that clients are talking to Pi-hole directly, as your previous top-clients did include various client IP addresses (and not only that of your router).
This would suggest that using Pi-hole as your router's upstream wouldn't be strictly required.
If that would be true, then you could also consider to point your router to a public DNS server of your choice in order to avoid closing a DNS loop.
As mentioned, it is caused by a request for resolving lb._dns-sd._udp.sanchez.is.
As sanchez.is seems to be your local domain, Pi-hole's CF would forward that request to your router. Your router wouldn't know the answer, and hence would forward it to its upstream, and if that happens to be Pi-hole, then Pi-hole's CF would forward that request to your router, and so on and so forth, ad infinitum or time-out.
In contrast, forwarded requests for known names like nas-1.sanchez.is can be answered by your router directly, thus avoiding the loop.
The interesting question would be why your client at 10.0.0.1 started to request lb._dns-sd._udp.sanchez.is.
As 10.0.0.1 does seem to be your router:
Do some of your clients still use yor router for DNS, instead of Pi-hole?
Maybe a static configuration on some device?
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.