How to rotate logs of Pi-hole hourly and log2ram issue?

Help Community Help
1

The issue I am facing:
It seems I have a high traffic home network, so the pihole.log gets several Gb in size during one day. I want to decrease the log size by running log rotate hourly for the pihole.log file instead of daily. I modified the file /etc/pihole/logrotateand changed daily to hourly. Is it enough? Or I should do some other modifications?

I also have a doubt that the logrotate runs at all for the Pi-hole files after I installed and enabled the log2ram, because I don’t see any new compressed rotated log files after that date for pihole.log. And my pihole.log file gets bigger and bigger until it consumes all my free space and I have to flush it manually. Is there any way to check the logs or status of running the logrotate job for the Pi-hole files?

**Details about my system: **
Debian 13 Trixie in LXC container, standard Pi-hole installation.

cat /etc/pihole/versions

CORE_VERSION=v6.1.4
CORE_BRANCH=master
CORE_HASH=cef7fd4b
GITHUB_CORE_VERSION=v6.1.4
GITHUB_CORE_HASH=cef7fd4b
WEB_VERSION=v6.2.1
WEB_BRANCH=master
WEB_HASH=cc1cc285
GITHUB_WEB_VERSION=v6.2.1
GITHUB_WEB_HASH=c0a237a6
FTL_VERSION=v6.2.3
FTL_BRANCH=master
FTL_HASH=88737f62
GITHUB_FTL_VERSION=v6.2.3
GITHUB_FTL_HASH=bc185680

cat /etc/pihole/logrotate
/var/log/pihole/pihole.log {
su root root
hourly
copytruncate
rotate 25
compress
delaycompress
notifempty
nomail
}

/var/log/pihole/FTL.log {
su root root
weekly
copytruncate
rotate 3
compress
delaycompress
notifempty
nomail
}

/var/log/pihole/webserver.log {
su root root
weekly
copytruncate
rotate 3
compress
delaycompress
notifempty
nomail
}

ls -lsh /var/log/pihole/
total 640K
8.0K -rw-r----- 1 pihole pihole 5.0K Sep  5 08:49 FTL.log
4.0K -rw-r----- 1 pihole pihole    2 Sep  5 08:29 FTL.log.1
4.0K -rw-r----- 1 pihole pihole 1.9K Sep  3 03:00 FTL.log.2.gz
4.0K -rw-r----- 1 pihole pihole 1.8K Sep  2 03:00 FTL.log.3.gz
196K -rw-r----- 1 pihole pihole 196K Sep  5 08:52 pihole.log
400K -rw-r----- 1 pihole pihole 398K Aug 28 03:00 pihole.log.6.gz
8.0K -rw-r----- 1 pihole pihole 4.2K Aug 31 07:17 pihole_updateGravity.log
4.0K -rw-r----- 1 pihole pihole  102 Sep  5 08:29 webserver.log
4.0K -rw-r----- 1 pihole pihole    2 Sep  5 08:29 webserver.log.1
4.0K -rw-r----- 1 pihole pihole  221 Aug 31 03:00 webserver.log.2.gz
4.0K -rw-r----- 1 pihole pihole  111 Aug 29 09:44 webserver.log.3.gz

df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/loop0      8.8G  6.4G  2.0G  77% /
none            492K  4.0K  488K   1% /dev
devtmpfs        4.0M     0  4.0M   0% /dev/tty
tmpfs            47G  2.1M   47G   1% /dev/shm
tmpfs            19G  120K   19G   1% /run
tmpfs            47G     0   47G   0% /tmp
tmpfs           5.0M     0  5.0M   0% /run/lock
log2ram         5.0G   44M  5.0G   1% /var/log
tmpfs           9.4G  8.0K  9.4G   1% /run/user/0

top
top - 08:55:57 up 44 min,  1 user,  load average: 0.00, 0.00, 0.13
Tasks:  26 total,   1 running,  25 sleeping,   0 stopped,   0 zombie
%Cpu(s): 10.0 us,  0.0 sy,  0.0 ni, 90.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   6144.0 total,   5177.7 free,     88.5 used,    923.0 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.   6055.5 avail Mem

What I have changed since installing Pi-hole:
I installed log2ram changed from the default configuration to use 5 Gb RAM for the logs to optimize the SSD writing and wear.

2

This very unusual in a home network.

Before starting to rotate logs, we need to know why the logs are so big.

This can be a result of a high number of devices (in this case log rotation will help you), but it can also be caused by a DNS loop or something else caused by a config issue.

Please upload a debug log and post just the token URL that is generated after the log is uploaded by running the following command from the Pi-hole host terminal:

pihole -d

or if you run your Pi-hole as a Docker container:

docker exec -it <pihole-container-name-or-id> pihole -d

where you substitute <pihole-container-name-or-id> as required.

3

I'm not sure if my network is considered to be big to generate so much log size or not.
I have a total of around 40 devices (phones, virtual machines, IP cameras, etc.).
However, today I see that my log size is just 5 Mb for 8 hours of work after logs flushing.
Could it be that today I run command pihole -f instead of manually deleting the log file as I usually did it.
Another idea is that sometimes my Internet connection goes down, could it lead to excessive logs writing?

The debug log is here https://tricorder.pi-hole.net/OK1dAHHD/
Thank you!

4

That depends on the apps/services on your devices.

Some apps will keep trying, again and again, to connect to a domain even after several failures. Sometimes these apps will increase the logs in case of Internet failure because the domain can't be reach until the Internet is back.

5

It seems that's exactly the behavior of my IP cameras. They keep trying to reach the remote control center every 5 seconds again and again. And they don't care if the network is down...

6

I had a similar experience when I turned off my Internet 3 months ago but left the router on for the LAN, with Pi-hole and a few other devices still on there but unable to reach anything external.

This wasn't a problem because I was using a mobile hotspot on a separate 172/12 subnet for the one computer that really did need external access.

After a couple of months (~ 8 weeks) I set up another Pi as a NAT gateway to the hotspot and used it to temporarily replace the router, to bring the original LAN subnet back online, to solve a specific problem with a device.

At this point the Pi-hole was back up and once again able to be used. However it wasn't working, FTL would not start and Unbound also no longer worked.

After some digging I found the card was full with ~16 GB of logs. Inspecting those found various devices, but especially Unbound itself and the Pi OS filling it with entries every few seconds, and this was going on for ~ 8 weeks with seemingly no rotation.

I wasn't able to get Unbound working so disabled it and switched to Cloudflare's DNS in Pi-hole, and I purged the logs as it was more important to get it back up and running in some usable form than diagnose what had happened.

It's now been working for around ~4 weeks in this slightly crippled form but has solved the problem here. Once I'm back on a normal connection in a few weeks I'll nuke the card, test it's reliable, and re-image with the latest Pi OS and a fresh build of Pi-hole and Unbound.

TL;DR – the OS itself as I've seen, and as you've seen various IoT devices, can be quite noisy if they don't get their own way and find a way to call home for some length of time, even with what you might describe as a normal network. The mere act of preventing them from their chatter can make them become very antisocial, hammering the DNS and gateway 24/7.

7

Thank you for sharing your experience! I have the same feeling that when Internet is down my devices go to panic mode and spam the DNS logs. I will run some tests next week by plugging off the cable. What surprises me that log rotation did not run in my case. Yet I'm not sure if it's related.

8

I haven't got that far yet, but you may need to check which elements of logrotate (e.g. hourly rotation) are supplied by the pi-hole software and which by the base OS on which it is installed.