Pi Hole diagnosis - Space issue

Help
1

Please follow the below template, it will help us to help you!

If you are Experiencing issues with a Pi-hole install that has non-standard elements (e.g you are using nginx instead of lighttpd, or there is some other aspect of your install that is customised) - please use the Community Help category.

Expected Behaviour:

I'm seeing alerts in the diagnosis that it is at 99% space. Specifically complaingin about the /var/log/pihole/pihole-ftl.log.

And yet when I look directly on the machine (LXC under Proxmox) it was (at the time I started) with 80% free. I cleared the alert as it seemed to be from a day ago, and it came back.

Machine is a 15gb disk, and I allow 45 days in the pihole-ftl.conf. With the pihole-ftl.db at about 6.9gb.

It's currently generating a debug log - which has essentially stopped it I think with high memory and high cpu usage.

Debug Token:

[Replace this text with the debug token provided from running pihole -d (or running the debug script through the web interface]
Waiting for token to be created.

2

Not exactly...

Pi-hole checks free disk space every time it writes to logs or database.

At the time the warning was generated, Pi-hole identified that your disk was 99% full. This check happened while Pi-hole was writing into /var/log/pihole/pihole-ftl.log. It doesn't mean necessarily this file was too big. It means the disk was almost full.

If the message is reappearing, some files are growing faster and the disk is already more 90% full. Maybe the files are not related to Pi-hole. You need to check the whole filesystem.

3

I've increased the space by 5gb and managed to get a token out of it: https://tricorder.pi-hole.net/zN4y918e/

Currently reporting 72% used. It was 80% used before. I can't see anything using it, and I'd like to see in the DB if it's ever been able to reduce the number of entries

4

6.9G is also still the size in your debog log, and that would be quite large for your long-term database, unless you had quite many very busy clients.

Run from your Pi-hole machine, what's the result of:

echo ">stats >quit" | nc localhost 4711

echo ">top-clients >quit" | nc localhost 4711
5 
domains_being_blocked 1133277
dns_queries_today 725947
ads_blocked_today 67651
ads_percentage_today 9.319000
unique_domains 3289
queries_forwarded 102637
queries_cached 555265
clients_ever_seen 33
unique_clients 33
dns_queries_all_types 725947
reply_UNKNOWN 420
reply_NODATA 50422
reply_NXDOMAIN 13903
reply_CNAME 402953
reply_IP 256010
reply_DOMAIN 1157
reply_RRNAME 2
reply_SERVFAIL 2
reply_REFUSED 0
reply_NOTIMP 0
reply_OTHER 0
reply_DNSSEC 19
reply_NONE 0
reply_BLOB 1059
dns_queries_all_replies 725947
privacy_level 0
status enabled


4711
0 188293 192.168.10.27 melkor
1 84823 192.168.10.55 sauruman
2 83063 192.168.10.45 drogo
3 76502 192.168.10.51 aragorn
4 76300 192.168.10.53 sauron
5 72904 192.168.10.155 NZ-5CG9495YGM
6 42200 192.168.10.42 fosco
7 33064 192.168.10.10 unifi-controller
8 17975 192.168.10.41 frodo
9 10457 192.168.10.175 Ugluk

I had a look at the db. The queries table. And it did look like it only had entires till Decemberish. Which is similar to the 45 days I’m happy to keep.

Are there other tables with lots of data as well?

I’m not sure why that top machine is so noisy. It hosts my home assistant and other home automation type containers.

6

33 clients isn't that many, but all of your top clients have been very busy during the last 24 hours. On average, I'd expect about 1,000 to 5,000 queries per client per day.

Let's see whether those counts would relate to only a few domains.
What's the result of

echo ">top-domains >quit" | nc localhost 4711

echo ">top-ads >quit" | nc localhost 4711
7 

 43862 aragorn
1 22639 sauruman
2 12743 plex.media
3 12375 mqtt
4 11687 cam-nvr
5 11302 traccar
6 11031 drogo
7 10792 lanwe
8 9989 manwe
9 9600 fosco
david@lanwe:~$  $



 20312 teams.events.data.microsoft.com
1 13660 mobile.events.data.microsoft.com
2 12809 self.events.data.microsoft.com
3 12773 us-mobile.events.data.microsoft.com
4 595 mask.icloud.com
5 577 aad.cs.dds.microsoft.com
6 508 xp.apple.com
7 483 g.live.com
8 477 firebaselogging-pa.googleapis.com
9 333 www.ojrq.net
david@lanwe:~$  $

Can’t I exclude local domain traffic? Or the really noisy machines? Ie I have influx (as influxdb) used a heap by a bunch of machines. If it possible to exclude that from being stored in the db and logged?

8

Would this discussion, which seems to be talking about excluding entries from the db (in version 6). Which I’m not yet on, help?

9

No, as that is about filtering queries for display in the webserver UI
(via webserver.api.excludeClients and webserver.api.excludeDomains).

Your top blocked domain counts seem reasonable (perhaps a little chatty on the microsoft side), but your top allowed domains are all for plain/non-dot local names?

Why would your clients request local names up to 30 times per minute?
Would they perhaps not cache DNS replies?

Please share the output of

dig aragorn

along with the respective query log lines:

sudo grep "Jan 17 <hh:mm>.*aragorn" /var/log/pihole/pihole.log

where <hh:mm> limits output to a specific hour and 1-minute interval (e.g. “Jan 17 07:10.*aragorn” to extract all entries for aragorn from today 07:10:00 to 07:10:59).

10

No my top domain and all those machines are against my domain name. It’s just been removed it from The logs. So for arguments sake use example.com.

All machines have the same domain . The Plex server is one further level down.

So:
aragorn.example.com
Plex.media.example.com

I also tend to run
*.internal.example.com - but they’re generally cname records to either drogo or fosco.

11

Are you saying you've manipulated the output from top-domains before sharing it here?
Or is that output you've shared above what was actually returned by echo ">top-domains >quit" | nc localhost 4711?

12

That text was manipulated. To removing my identifying domain.

13

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.