Saving query logs for long-term analysis

First, install the build dependencies. I don't like to fight with ./configure to figure out what all needs to be installed, so I just sledgehammer it with:

sudo apt-get build-dep logrotate

Get the 3.11.0 source, extract, and build it:

wget https://github.com/logrotate/logrotate/releases/download/3.11.0/logrotate-3.11.0.tar.gz
tar xf logrotate-3.11.0.tar.gz
cd logrotate-3.11.0
./configure --with-acl --with-state-file-path=/var/lib/logrotate/status --prefix=/usr
make && sudo make install

The --with-state-file-path part of the ./configure line is important because that's where Debian keeps its status file. Be warned that the --prefix setting will overwrite what is installed by apt-get.

Stand up, do 10 jumping jacks. It takes about a minute on my Raspberry Pi B+, the first one.

Now, give it a try again:

sudo /usr/sbin/logrotate -d /etc/pihole/logrotate

reading config file /etc/pihole/logrotate
olddir is now /mnt/dns-backup
Reading state from file: /var/lib/logrotate/status
Allocating hash table for state file, size 64 entries
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state

Handling 1 logs

rotating pattern: /var/log/pihole.log  after 1 days (no old logs will be kept)
olddir is /mnt/dns-backup, empty log files are not rotated, old logs are removed
considering log /var/log/pihole.log
  Now: 2017-02-27 13:24
  Last rotated at 2017-02-24 00:00
  log needs rotating
rotating log /var/log/pihole.log, log->rotateCount is 0
Converted ' -%Y%m%d' -> '-%Y%m%d'
dateext suffix '-20170227'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding logs to compress failed
removing /mnt/dns-backup/pihole.log-20170220.gz
removing old log /mnt/dns-backup/pihole.log-20170220.gz
removing /mnt/dns-backup/pihole.log-20170221.gz
removing old log /mnt/dns-backup/pihole.log-20170221.gz
removing /mnt/dns-backup/pihole.log-20170222.gz
removing old log /mnt/dns-backup/pihole.log-20170222.gz
copying /var/log/pihole.log.tmp to /mnt/dns-backup/pihole.log-20170227
Not truncating /var/log/pihole.log.tmp
removing tmp log /var/log/pihole.log.tmp
removing old log /mnt/dns-backup/pihole.log-20170223.gz

You'll notice a ghastly thing! We need to have a rotate directive, because otherwise, logrotate will delete our old log files! Imagine my terror when I momentarily forgot that the -d option prevents logrotate from actually taking action, thereby removing my log files from previous days!

So, let's change our logrotate config to include an absured rotate directive. 10 years ought to be enough for anyone.

/var/log/pihole.log {
	su root root
	daily
	rotate 3650
	copytruncate
	renamecopy
	compress
	delaycompress
	notifempty
	nomail
	olddir /mnt/dns-backup
	missingok
	dateext
	dateformat -%Y%m%d
}

Run a trial run again and inspect the output to make sure it's doing what you want:

sudo /usr/sbin/logrotate -d /etc/pihole/logrotate

I think this looks OK, because I don't see anything getting deleted now:

reading config file /etc/pihole/logrotate
olddir is now /mnt/dns-backup
Reading state from file: /var/lib/logrotate/status
Allocating hash table for state file, size 64 entries
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state

Handling 1 logs

rotating pattern: /var/log/pihole.log  after 1 days (3650 rotations)
olddir is /mnt/dns-backup, empty log files are not rotated, old logs are removed
considering log /var/log/pihole.log
  Now: 2017-02-27 13:30
  Last rotated at 2017-02-24 00:00
  log needs rotating
rotating log /var/log/pihole.log, log->rotateCount is 3650
Converted ' -%Y%m%d' -> '-%Y%m%d'
dateext suffix '-20170227'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding logs to compress failed
copying /var/log/pihole.log.tmp to /mnt/dns-backup/pihole.log-20170227
Not truncating /var/log/pihole.log.tmp
removing tmp log /var/log/pihole.log.tmp

So, we'll see how this runs tonight!

1 Like