Running out of disk space

This is not really a question regarding Pi-hole, but maybe someone can help me out here.

I got these two warnings in Pi-hole diagnostics this morning:

Disk shortage (/etc/pihole/pihole-FTL.db) ahead: 96% used
/etc/pihole: 29.7GB used, 30.8GB total

Disk shortage (/var/log/pihole/FTL.log) ahead: 96% used
/var/log/pihole: 29.7GB used, 30.8GB total

So I checked with df what was going on

Filesystem      Size  Used Avail Use% Mounted on
udev            660M     0  660M   0% /dev
tmpfs           185M  3.9M  181M   3% /run
/dev/mmcblk0p2   29G   27G  1.1G  97% /
tmpfs           924M  7.1M  917M   1% /dev/shm
tmpfs           5.0M   20K  5.0M   1% /run/lock
/dev/mmcblk0p1  510M   65M  446M  13% /boot/firmware
/dev/sda1       4.6T  550G  3.8T  13% /mnt/timemachine
tmpfs           185M     0  185M   0% /run/user/1000

So apparently /dev/mmcblk0p2 is used up 97%. But when I do a du, the numbers don't add up.

sudo du -smh /*  | sort -nr
574M	/etc
551G	/mnt
280K	/opt
210M	/boot
56K	/tmp
40K	/root
26M	/home
16K	/lost+found
7.1M	/dev
4.0K	/srv
4.0K	/media
3.9M	/run
2.7G	/var
1.9G	/usr
0	/vmlinuz.old
0	/vmlinuz
0	/sys
0	/sbin
0	/proc
0	/lib
0	/initrd.img.old
0	/initrd.img
0	/bin

What am I missing here? There should be roughly 20G of free space…

You can try to search for large files using the find command ( you may need to install it if its not already there ).

sudo find / -type f -size +50M -exec ls -Shl {} +

This will look for a files over 50 megabytes and then post them using the ls command. You can change the +50M to anything you like ie +1G would be 1 gigbyte files etc.

That's the weird thing. The only big files that show up are from the sparsebundle, like:

1.2G Jul 30 03:28 '/mnt/timemachine/MacBook Pro.sparsebundle/bands/ff

And these are written to a different drive:

/dev/sda1       4.6T  550G  3.8T  13% /mnt/timemachine

Maybe I need to wait until the Time Machine Backup finishes and restart the pi …

It looks like you're not getting a full picture from the command you used. Try running the interactive du below, this will sort by size and let you explore to see where the space is being used. You can install ncdu if it's not present.

sudo ncdu --si /
2 Likes

Interesting tool. I was not aware of it. Always a good day when your learn something new. :slight_smile:

Hmm, nope, still the same:

  761.0 GB  [##############################] /mnt
    2.8 GB  [                              ] /var
    2.0 GB  [                              ] /usr
  602.9 MB  [                              ] /etc
  219.4 MB  [                              ] /boot
   26.4 MB  [                              ] /home
    7.5 MB  [                              ] /dev
    4.2 MB  [                              ] /run
  286.7 KB  [                              ] /opt
   57.3 KB  [                              ] /tmp
   41.0 KB  [                              ] /root
e  16.4 KB  [                              ] /lost+found
e   4.1 KB  [                              ] /srv
e   4.1 KB  [                              ] /media
    0.0  B  [                              ] /sys
.   0.0  B  [                              ] /proc
@   0.0  B  [                              ]  initrd.img.old
@   0.0  B  [                              ]  initrd.img
@   0.0  B  [                              ]  vmlinuz.old
@   0.0  B  [                              ]  vmlinuz
@   0.0  B  [                              ]  sbin
@   0.0  B  [                              ]  lib
@   0.0  B  [                              ]  bin

But I noticed some errors when using du:

du: cannot access '/proc/83611/task/83611/fd/4': No such file or directory
du: cannot access '/proc/83611/task/83611/fdinfo/4': No such file or directory
du: cannot access '/proc/83611/fd/3': No such file or directory
du: cannot access '/proc/83611/fdinfo/3': No such file or directory

I am not sure if this is of any relevance.

Please elaborate on the involvement of Time Machine with this Pi. You are backing the Pi up to a Time Machine drive? Of, you are using the Pi as a Time Machine server?

You can run sudo lsof -p 83611 It may provide you some info on that process. As you said, not sure if its relevant or not but its more info.

It is odd that some of those folders show 0 bytes when they should clearly have files.

Yes, exactly this.

I mounted a 5 TB External hard drive to /mnt/timemachine and back up my MacBook. I have this setup running for a couple of months now. And never had this warning before.

Yesterday, my old hard drive died, I removed it and added a new one. Nothing besides that changed. And now I'm just doing the initial backup (~1.2 TB).

I mounted the new drive to the same path as I did with the old one. The only change, of course, was the UUID in /etc/fstab.

UUID=258cf132-edb9-4453-b7fe-82550018e9a4	/mnt/timemachine/	ext4    defaults,noatime	0	1

Time Machine can do some strange things with files (sparse bundles, snapshots, etc.).

Given that your problem started when you replaced a drive, my inclination would be that the two events are related.

Let Time Machine do its thing for a few days and see if the issue resolves.

For what it's worth, I also run a MacBook Pro, and I just hook the backup drives directly to a USB hub connected to the Mac. The Mac software does the work without needing a separate Time Machine server. I have two drives connected most of the time, and Time Machine rotates through them each time a backup is done.

I finally figured it out. Apparently, some data got written to the underlying root partition /dev/mmcblk0p2 instead of the mounted one /dev/sda1.

I figured this out after unmounting sda1 and still have the sparse bundle folder in /mnt/timemachine.

After deleting the redundant folder and checking/verifying the backup, everything is back to normal.

Thank you all for your help!

I know you figured it out, but I would suggest that any mount point directories be set to read-only. Once a device partition is mounted to the mount point, the permissions of the mounted file system take over.

I learned about this after having a drive fill up, and I could not figure out why. After some time digging around, I discovered that at one point I had failed to mount an external drive the path /media/temp that I had created. Copied some data to /media/temp, thinking it was going to my external drive. Once I figured it out, read that mount points could be marked read-only, and that they would adopt the mounted devices permissions. Lesson learned for me.

1 Like

Thank you for this tip.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.