Log Full on /var/log on a constant basis, need help finding how to remount it

Help Community Help
1

Expected Behaviour:

Logs should rotate or clear as needed, /var/log should write to disk

Actual Behaviour:

/var/log is filling up quickly, PiHole is reporting it has less disk space than the drive it is installed on

Debug Token:

https://tricorder.pi-hole.net/o26XSNmU/

I searched online and found this thread ( Constantly getting a log full error ) which describes the same issue I am having. However, when I check fstab, I do not see a mount for /var/log (although I may be missing it).

The setup is an Orange Pi running Debian off of a 32 GB SD card, so there should be plenty of space. What I am not sure of at this time is how to check how / where /var/log is mounted, since it is not in fstab. I also have Cockpit installed on the same platform, and the Storage module is not showing a mount there, either.

Thank you for your help!

2

Looking at my diagnosis, I can see:

*** [ DIAGNOSING ]: Disk usage
(snip)
/dev/zram1 47M 29M 15M 67% /var/log

So it's definitely the same issue - the trick now is how to turn this off.

This is a Debian server, if that will help point in the right direction?

3

Hi @GJSchaller,

As far as I know there isn't a specific mount point for /var/log by default, but you can check with:

cat /etc/mtab | grep log

if it's mounted or if something is mounted on /var/log. If there is a mount, you can remount it with

sudo mount -o remount,rw /partition/identifier /mount/point

Or something. I don't use or know anything about zram.

Also, the Arch Wiki is usually invaluable with this kind of thing.

4

Thank you for the pointer - I see lot of information on how to set up a zram swap manually. Unfortunately, I don't see any guides on how to disable one that's been configured for boot.

I'll keep digging to see if I can figure out how to do this. For future searches, this is using the Orange Pi image for Debian, in case anyone else is looking for this.

1 Like
5

As mentioned above, Debian doesnt mount the logs to zram OOTB.
Does below one show you're running Armbian by any chance?

hostnamectl

If so:

This can be disabled by editing /etc/default/armbian-zram-config and changing the line

ENABLED=true

I moved this thread to the community section as this is not a Pi-hole issue.

6

It's from the Orange Pi downloads, so I suspect it's a fork of Armbian. When I checked for that file, I didn't see the exact file, but I did see orangepi-zram-config

The trick is, if I disable it, I think I'll also be disabling zram for Swap? Is that desirable?

7

If its Armbian, I'd recommend asking at their support channels.

8

Your link pointed me in the right direction. I am also realizing that the reason this is in place is to prevent wear & tear on the storage device by using RAM for a high-use volume - so it's likely not a good idea to disable it altogether.

Operating System: Debian GNU/Linux 12 (bookworm)
Kernel: Linux 6.1.31-sun50iw9
Architecture: arm64

Looks like I may want to grab a device with more RAM if this becomes a larger issue down the road.

9

They are exaggerating the wear & tear for normal use.
Other SBC related distros based on Debian dont have a zram or tmpfs mount for the logs ... OOTB.
Including Raspbian for the most famous of SBC's.
I have two Raspi's running Pi-hole for over eight years now without a glitch and still running on the original SD cards.
Those solutions to store logs in RAM most often cause more pain than gain.

Plus so what if the SD card gets destroyed by excessive wear & tear.
They are cheap and I can get them at my local grocery store.
Just make sure you backup the important data ... which you should do in any case.

It also helps to get a spacious SD card to spread the wear & tear.

If want to stick with zram for the logs, you could try lowering below rotate 5 directive to save some disk space:

$ cat /etc/pihole/logrotate
/var/log/pihole/pihole.log {
    su root root
    daily
    copytruncate
    rotate 5
    compress
    delaycompress
    notifempty
    nomail
}

/var/log/pihole/FTL.log {
    su root root
    weekly
    copytruncate
    rotate 3
    compress
    delaycompress
    notifempty
    nomail
}
/var/log/pihole/webserver.log {
su root root
weekly
copytruncate
rotate 3
compress
delaycompress
notifempty
nomail
}

FYI:

$ man logrotate.conf
[..]
       rotate count
              Log files are rotated count times before  being  removed  or
              mailed  to  the  address  specified in a mail directive.  If
              count is 0, old versions are removed  rather  than  rotated.
              If count is -1, old logs are not removed at all, except they
              are affected by maxage (use with caution, may waste  perfor‐
              mance and disk space).  Default is 0.
10

I favor getting a second SD card for backup and once you have everything configured to your liking, copy the entire SD content to the backup SD using the Linux disk destroyer (dd) or Disk Imager for Windows.

Plus the occasional Pi-hole Teleporter backup via shell or the webGUI.

1 Like
11

The original thread was mine.

I disabled some of the logs, increased the size of zram and got it to rotate less instances of the log.

That stopped my issue.

Pihole is still running in the same orange pi with the same sdcard with no issues.