Saving query logs for long-term analysis

I decided to configure logrotate with the olddir set to an NFS mount to my NAS. Moreover, I'll use the dateext directive to configure logrotate to save the log backups with a date instead of a number indicating how many days ago the log was rotated out.

My steps, writing as I go, with intent to turn this into a HOWTO. I performed this while SSH'd into my Raspberry Pi B+ running Pihole v2.12.1 on Raspbian Jessie.

Before doing anything here, ensure that both Pihole and apt are up to date:

pihole -up
sudo apt-get update && sudo apt-get upgrade

Preparing the NFS backup directory

Verifying NFS accessibility

I created the NFS share on my NAS, which will be addressed for the purposes of this write-up as mynas.local:/dns-backup. You will want to use a static IP address for the remote server instead of a hostname if your Pihole is doing DHCP. Mine is not.

sudo mount mynas.local:/dns-backup /mnt/dns-backup -o nolock
dd if=/dev/zero of=/mnt/dns-backup/zeros count=2 bs=1M

Check to see if the file exists. If you didn't get an error after the mount and the file exists, you're good to go. Unmount it.

sudo umount /mnt/dns-backup

Mounting NFS

I decided to use autofs instead of putting the mount entry in /etc/fstab because the latter can delay system startup or cause a backup malfunction if for some reason the share cannot be mounted. I obviously want my Pihole to come back up as quickly as possible after going down for any reason! This Unix & Linux Stack Exchange question swayed me.

sudo apt-get autofs

You'll need to create /etc/auto.master.d, which is configured to load automatically but was not automatically created for me.

sudo mkdir -p /etc/auto.master.d

Next, create your autofs map file and the file containing the mount point and the mount target

echo -e "/mnt\t\t/etc/auto.master.d/mnt.map" | sudo tee /etc/auto.master.d/mnt.autofs
echo -e "dns-backup\t\t-fstype=nfs,nolock,soft,noexec,nosuid\t\tmynas.local:/dns-backup" | sudo tee /etc/auto.master.d/mnt.map

You may need to adjust those NFS options to meet your own needs.

Start autofs with

sudo service autofs start

then verify that you see the zeros file you created earlier when you do

ls /mnt/dns-backup

If you see the zeros file, then your mount is correctly configured! If not, tail /var/log/syslog to see any errors. It took me a good hour to find just the right syntax for the map file. I saved you an hour :wink:

Ensure that it sticks

This is a great time to reboot your Pihole server to ensure that the mounts you've created will come back up when on restart.

sudo reboot

Give it a few seconds and reconnect.

ls /mnt/dns-backup

If you see the zeros file, then your mount is fully prepared and ready to receive data.

Adjusting logrotate configuration

Why Pihole puts its logrotate configuration into /etc/pihole/logrotate instead of /etc/logrotate.d/pihole is beyond me, but we'll go with it for now.

Make /etc/pihole/logrotate look like this:

/var/log/pihole.log {
	su root root
	daily
	copytruncate
	compress
	delaycompress
	notifempty
	nomail
	olddir /mnt/dns-backup
	missingok
	dateext
	dateformat -%Y%m%d
}

logrotate is executed daily by cron, so you'll basically have to wait until midnight to see if this working correctly. You could run logrotate manually with sudo /usr/sbin/logrotate -s /tmp/statefile /etc/logrotate.conf but you'd have to run it again in 24 hours or manually change the timestamps to test it! I'm lazy and I'm not doing that.

With any luck, come midnight the day after you've set this up, you'll have a dated log file in your NFS share! After a couple of days, you'll have several, all but one of which will be compressed. Note that I removed the rotate directive because I want to store everything forever and I have space warnings set up for my NAS. If you have a lot of DNS traffic, be mindful of what you're doing and maybe consider storing only a year's worth of data.

1 Like