Disk shortage (/var/log/pihole/FTL.log) ahead: 99% used

Yeah8302 · November 2, 2022, 8:13pm

Disk shortage (/var/log/pihole/FTL.log ) ahead: 99% used
/var/log/pihole: 314.6MB used, 314.6MB total

I have been filling up the pihole.log for the past several months and have little to no knowledge with Pi-hole, Raspberry Pi or it's OS. I tried reading thru the other Community Help thread, but could not see how to resolve my problem. I do use Log2RAM and keep changing the disk size to try to give some headroom, but the logging fills it up super quick. I think I understand that LogRotate fails because of the size that pihole.log grows to before the rotation schedule is executed.

pi@raspberrypi:~ $ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"

Raspberry Pi 4 4GB 32MB

pi@raspberrypi:~ $ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 29G 6.2G 22G 23% /
devtmpfs 1.8G 0 1.8G 0% /dev
tmpfs 1.9G 100M 1.8G 6% /dev/shm
tmpfs 1.9G 201M 1.7G 11% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/mmcblk0p1 253M 49M 204M 20% /boot
log2ram 300M 300M 0 100% /var/log
tmpfs 384M 0 384M 0% /run/user/999
tmpfs 384M 4.0K 384M 1% /run/user/1000

Debug Token:

Thanks for your help!

Bucking_Horn · November 2, 2022, 8:38pm

With a log file growing quickly out of proportion, I'd usually look for hints of a DNS loop in your debug log. But alas, your debug log has been truncated - likely, because your log2ram file system was out of space already.

Could you clear the logs and provide another token, please?

Yeah8302 · November 2, 2022, 9:59pm

Thanks for the quick response. I reconfigured pi-hole and started actually looking at the huge number of queries on the dashboard and found that I had explicitly blacklisted ExpressVPN, which I am using in a dual-router configuration! As soon as I deleted it from the domain list, the queries stop piling up. So now the pihole.log seems more reasonable(?), growing at about 1MB/hr. I'll watch it over the next few days and see how the log rotation is working out.

Here is the token: https://tricorder.pi-hole.net/qJbwwwiD/

What does this mean? It did not correct itself after reconfiguring pi-hole.
*** [ DIAGNOSING ]: Dashboard headers
[✗] Web interface X-Header: X-Header does not match or could not be retrieved.
HTTP/1.1 301 Moved Permanently
Location: https://localhost/admin/
Date: Wed, 02 Nov 2022 21:55:50 GMT
Server: lighttpd/1.4.53

Thanks!

jfb · November 3, 2022, 12:11am

It means you have your lighttpd web admin configured for SSL, via your lighttpd external config file.

Bucking_Horn · November 3, 2022, 8:43am

I don't think there is a failure with log rotation.
While it intends to prevent log files from growing out of proportion, it does so by checking, moving and removing log files on a regular basis. Specifically, it does not monitor remaining space continuosly.

So if you haven't explicitly reconfigured it, most Linux distros will run logrotate as a daily cron job.
In your case, available storage space likely is exhausted before logrotate is even starting its daily processing.

Your individual log growth rate would depend on the number of clients and their level of DNS activity.
That means I can't give you a final verdict, but while 1 MB per hour still seems a bit on the high end, it could be reasonable for your network.

Your debug log shows you've enabled Pi-hole's Conditional Forwarding, but your local domain mrlHomeAX differs from the one distributed by your router's DHCP server mrlHomeAX.local:

*** [ DIAGNOSING ]: Discovering active DHCP servers (takes 10 seconds)
   Scanning all your interfaces for DHCP servers
   
   * Received 323 bytes from eth0:192.168.3.1
     Offered IP address: 192.168.3.197
     DHCP options:
      Message type: DHCPOFFER (2)
      domain-name: "mrlHomeAX.local"
      dns-server: 192.168.3.1
      dns-server: 192.168.3.254
      dns-server: 192.168.3.1
      router: 192.168.3.1
      --- end of options ---

Adjust any of those local domain names to make them match.
In your case, it would be preferred if you changed the name your router is using, as .local is reserved for mDNS usage and shouldn't be used with DNS.

In addition, your router is distributing its own IP as local DNS server (twice), and together with Pi-hole's CF, this would be a precondition to potentially close a DNS loop (depending on your router's upstream DNS server configuration), and/or would allow clients to completely by-pass Pi-hole via your router.

Pi-hole must be the sole DNS server for your clients, so configure your router to just distribute Pi-hole as local DNS server.

Then verify that your router is not using Pi-hole as upstream DNS server, if you'd want to avoid a partial DNS loop.

Yeah8302 · November 3, 2022, 4:21pm

Sorry for all the screen slips--I'm a visual type.
This is my raspberry pi ethernet eth config with pi-hole being 192.168.3.254. I'm also using unbound.
My main router is 3.1 (I changed the router domain name to mrlHomeAX per your comments).

What should the DNS server and search be?
Is the current config the reason I am seeing 3.1 as DNS server? Should it be 192.168.3.254? What in that case would the DNS Search field be?
I set the WAN DNS to pi-hole (3.254) because it seemed like some clients (my phone) were bypassing pi-hole when I configured it to some commercial DNS (Quad-9 or Google). I also have guest wi-fi (192.168.101.x assigned by Asus) used for IOT devices, which I don't want/need to run thru pi-hole. There is no DNS field for these guest networks, so I assume it uses the WAN DNS configuration.
My LAN DNS is directed to pi-hole. Should my WAN DNS point to pi-hole or not?

Thanks again for all your help.

nprampage · November 3, 2022, 7:53pm

Gonna offer some input here, not a dev, just a user like you. LAN DNS should be Pihole, WAN DNS might need to be a public DNS server like Google or 9.9.9.9 (Quad9), as some routers don't like having their WAN DNS pointed to Pihole (mine, a Synology 2600, most definitely is one of those!).

I think your DNS Search Field needs to have the ".local" part removed. The "home.arpa" suffix is a recognized setting for this, there are a few others that are considered acceptable as well.

system · November 24, 2022, 7:54pm

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.