# Implement logrotation for /var/log/pihole.log

**URL:** <https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266>\
**Category:** Implemented\
**Created:** [January 12, 2017, 7:32am UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266 "2017-01-12T07:32:10Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![jpgpi250](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/jpgpi250/32/286_2.png) [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)\
**Post date:** [January 12, 2017, 7:32am UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/1 "2017-01-12T07:32:10Z")

</div>

This idea came to me when I was working on [this](https://discourse.pi-hole.net/t/send-logs-before-flushing/1254) topic. Since log rotation is part of almost all linux distros, I checked if this could be used, ...  
Solution:

- Create /etc/.pihole/advanced/logrotate (the instructions for the rotation)  
`/var/log/pihole.log {`  
`daily`  
`copytruncate`  
`rotate 1`  
`nocompress`  
`}`
- Modify /opt/pihole/piholeLogFlush.sh  
echo -n "::: Rotating /var/log/pihole.log ..."  
\*\*#\*\*echo " " \> /var/log/pihole.log  
**/usr/sbin/logrotate --force /etc/.pihole/advanced/logrotate**  
echo "... done!"

considerations:

- There aren't many log rotate directives, as they will be ignored by `--force`
- The log will be available for the next 24 hours (`rotate 1`)
- I opted for `nocompress`, this way the log remains readable, without additional actions
- I tested various options to initialize the log after rotation, `copytruncate` seems to work best. This implies the log must exist! I tested this with a `postrotate script` (echo " " \> /var/log/pihole.log) but that doesn't work, the new log remains empty.
- You can't use the systems default logrotation scheme, since daily logrotation occurs at 06h25.

tested on pihole v.2.11.2, **pihole flush** aswell as **cron**

the developers could make yesterdays log available on the web interface by parsing the log /var/log/pihole.log.1

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 12, 2017, 7:20pm UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/2 "2017-01-12T19:20:16Z")

</div>

We are actually considering implementing this like for this quite some time now. However, we have to ensure first that it will work nicely on all platforms we support, which is quite a time-consuming task. And it got a little out of focus while working on `FTL`.

Let me ask you some questions about your precise suggestion:

> [@jpgpi250](#):
>
> The log will be available for the next 24 hours (rotate 1)

If the user flushes it twice in a row, the log from the "day before" is gone, I suppose (that is how I'd expect it to work)?

> [@jpgpi250](#):
>
> I tested various options to initialize the log after rotation, copytruncate seems to work best. This implies the log must exist!

What happens otherwise?

We could add `missingok` in case.

> [@jpgpi250](#):
>
> You can't use the systems default logrotation scheme, since daily logrotation occurs at 06h25.

Timing might actually be an issue. We could do it with `cron` at midnight. What if the Pi is not running at midnight? Of course, there is the `@reboot` statement with `cron`: do you know if there is an option to do the logrotation only if it has not been done _today_? I hope you get what I mean. Pi-hole's not running 24h are seeing this issue and it would be nice if this approch would provide an elegant way to mitigate that problem.

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 12, 2017, 7:44pm UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/3 "2017-01-12T19:44:44Z")

</div>

[https://github.com/pi-hole/pi-hole/pull/1130](https://github.com/pi-hole/pi-hole/pull/1130)

---

<div class="post-metadata">

**Author:** ![jpgpi250](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/jpgpi250/32/286_2.png) [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)\
**Post date:** [January 12, 2017, 8:56pm UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/4 "2017-01-12T20:56:36Z")

</div>

The answer to your questions:

- If the user flushes twice in a row, the log from the "day before" is gone: This is no different from the current situation, the current **pihole flush** command removes all the logging. I might even suggest to remove the 'flush' button from the web interface...

- This implies the log must exist: If the log does not exist, it cannot be rotated and no new log will be created. Adding **missingok doesn't solve this** , it just suppresses the error message, **witch I would not advise** , an error message indicates something is wrong.  
To overcome the problem of a non existing log, add this to /opt/pihole/piholeLogFlush.sh. Tested this, works fine.  
/usr/sbin/logrotate --force /etc/.pihole/advanced/logrotate  
**if [! -f /var/log/pihole.log]; then**  
**echo " " \> /var/log/pihole.log**  
**service dnsmasq restart**  
**fi**  
Warning: I choose **copytruncate** because logging didn't resume using other options (always had to restart dnsmasq). My log is about 600Kb in size, I've got no idea what will happen if the log is several Mb in size. This should be tested!

- Time might actually be an issue: I've modified /opt/pihole/piholeLogFlush.sh, witch is already triggered by cron at midnight (pihole flush). If the pi isn't running 24/7, there is no difference from the current situation e.g. no rotation.  
To my knowledge, there is no logrotate option to catch up missed events. You can find all logrotate options [here](https://linux.die.net/man/8/logrotate).

- I would advise against the @reboot cron option, als the log will be rotated at every reboot, witch I (and others) often do when I'm testing things. You could consider implementing @reboot ( **you need to add the root user** - tested this) with a script that detects a non rotated log and rotate it only if this is detected. Example:  
`line=$(head -n 1 /var/log/pihole.log)`  
`date=$(date +"%b %d")`  
`if [[${line} != *$date*]];then`  
`	/usr/sbin/logrotate --force /etc/.pihole/advanced/logrotate`  
`fi`

- I think the **notifempty** (saw this on github) directive is ignored by the command line **force** option.

- The **nomail** instruction doesn't do anything, if nothing is configured, the mail you can recieve (see chapter 4.8 of [this](http://users.telenet.be/MySQLplaylist/pi-hole.pdf) manual) comes from the cron daemon.

- **rotate 5** may be over the top. Even most system logs (see /etc/rotate.d/rsyslog) are rotated 4 times only.

- You should probably check the precense of the **logrotate** package in **install.sh**

to summarize all this (tested)  
**/opt/pihole/piholeLogFlush.sh** should look like this  
`#!/usr/bin/env bash`  
`# Pi-hole: A black hole for Internet advertisements`  
`# (c) 2015, 2016 by Jacob Salmela`  
`# Network-wide ad blocking via your Raspberry Pi`  
`# http://pi-hole.net`  
` # Flushes /var/log/pihole.log`  
` #`  
`# Pi-hole is free software: you can redistribute it and/or modify`  
`# it under the terms of the GNU General Public License as published by`  
`# the Free Software Foundation, either version 2 of the License, or`  
`# (at your option) any later version.line=$(head -n 1 /var/log/pihole.log)`

`# detect the reboot parameter from cron`  
`if [[$@ == **reboot**]];then`  
`	date=$(date +"%b %d")`  
`	if [[${line} != *$date*]];then`  
` /usr/sbin/logrotate --force /etc/.pihole/advanced/logrotate`  
`	fi`  
`else`  
`	echo "::: Rotating /var/log/pihole.log ..."`  
`	/usr/sbin/logrotate --force /etc/.pihole/advanced/logrotate`  
`	echo "::: ... done!"`  
`fi`  
`#catch a non existing log`  
`if [! -f /var/log/pihole.log]; then`  
` echo " " > /var/log/pihole.log`  
`	service dnsmasq restart`  
`fi`

**/etc/cron.d/pihole** should look like this:  
`# Pi-hole: A black hole for Internet advertisements`  
`# (c) 2015, 2016 by Jacob Salmela`  
`# Network-wide ad blocking via your Raspberry Pi`  
`# http://pi-hole.net`  
`# Updates ad sources every week`  
`#`  
`# Pi-hole is free software: you can redistribute it and/or modify`  
`# it under the terms of the GNU General Public License as published by`  
`# the Free Software Foundation, either version 2 of the License, or`  
`# (at your option) any later version.`  
`#`  
`# This file is under source-control of the Pi-hole installation and update`  
`# scripts, any changes made to this file will be overwritten when the softare`  
`# is updated or re-installed. Please make any changes to the appropriate crontab`  
`# or other cron file snippets.`

`# Pi-hole: Update the ad sources once a week on Sunday at 01:59`  
`# Download any updates from the adlists`  
`59 1 * * 7 root PATH="$PATH:/usr/local/bin/" pihole updateGravity`

`# Pi-hole: Update Pi-hole! Uncomment to enable auto update`  
`#30 2 * * 7 root PATH="$PATH:/usr/local/bin/" pihole updatePihole`

`# Pi-hole: Flush the log daily at 00:00 so it doesn't get out of control`  
`# Stats will be viewable in the Web interface thanks to the cron job above`  
`00 00 * * * root PATH="$PATH:/usr/local/bin/" pihole flush >/dev/null`

`@reboot root /opt/pihole/piholeLogFlush.sh reboot`

notice the reboot parameter on the @reboot line, it makes piholeLogFlush check the date of the pihole log (first line)  
notice the redirection on the midnight (00 00) line, a mail will only be sent if somethings wrong.

**/etc/.pihole/advanced/logrotate** should look like this:  
`/var/log/pihole.log {`  
`	daily`  
`	copytruncate`  
`	rotate 4`  
`	compress`  
`	delaycompress`  
`	nomail`  
`}`

to make life easy for you, I've made an online [archive](http://users.telenet.be/MySQLplaylist/logrotation.rar) with the required files.

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 13, 2017, 10:09am UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/5 "2017-01-13T10:09:37Z")

</div>

I see you edited the post 12 times (last time 7min ago). I've read this version, so please make anew post if you implement further changes

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 13, 2017, 10:15am UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/6 "2017-01-13T10:15:52Z")

</div>

> [@jpgpi250](#):
>
> Warning: I choose copytruncate because logging didn't resume using other options (always had to restart dnsmasq). My log is about 600Kb in size, I've got no idea what will happen if the log is several Mb in size. This should be tested!

I have a log that is several MB in size (about one dozen quite active clients). It works without problems. `dnsmasq` keeps the log file open while running and therefore if you move the file it will happily write into the moved file instead of the new one (as the pointer to the virtual memory location does not change and renaming does not change where the file is physically located but only its name).

> [@jpgpi250](#):
>
> I would advise against the @reboot cron option, als the log will be rotated at every reboot, witch I (and others) often do when I'm testing things.

I tested this already (you can see it is in the code of the pull request): The log has been flushed at midnight. When I reboot, `logrotate` states that the file is not older than one day and simply skips the rotation procedure without having us to worry about it. Tried with three reboots within an hour - no rotation happened for me - did you see something different that would require implementing the changes you suggested (with checking the date)?

EDIT: You can see this when running the command manually with the verbose option:

```auto
# /usr/sbin/logrotate -v /etc/.pihole/advanced/logrotate
reading config file /etc/.pihole/advanced/logrotate

Handling 1 logs

rotating pattern: /var/log/pihole.log after 1 days (5 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/pihole.log
  log does not need rotating

```

(note that the log was several hundred KB is size at this time, so the line about empty log files is only informative)

---

<div class="post-metadata">

**Author:** ![jpgpi250](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/jpgpi250/32/286_2.png) [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)\
**Post date:** [January 13, 2017, 10:27am UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/7 "2017-01-13T10:27:18Z")

</div>

I don't think your @reboot line in cron is doing anything. I've tested this, the way your github modifications specify and get a log entry (error - syslog). If you add the user (root), the log is flushed when the system is rebooted, actually no different from pihole flush.  
The command line --force option actually instructs logrotate to rotate, no matter what the conditions are.  
To test this, I used yesterdays log and simply copied it into todays log, I think I've covered all scenarios, even a non existing log (that should never occur).

@reboot **root** /opt/pihole/piholeLogFlush.sh reboot

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 13, 2017, 10:30am UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/8 "2017-01-13T10:30:00Z")

</div>

> [@jpgpi250](#):
>
> The command line --force option actually instructs logrotate to rotate, no matter what the conditions are.

The line `--force` is **not** there!

 ![](https://discourse.pi-hole.net/uploads/default/original/3X/9/c/9c9e842a415007961c000c9a0a17aa45c54d1b1d.png)

---

<div class="post-metadata">

**Author:** ![jpgpi250](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/jpgpi250/32/286_2.png) [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)\
**Post date:** [January 13, 2017, 10:36am UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/9 "2017-01-13T10:36:00Z")

</div>

Sorry, I missed that, will need to test again, however, NOT seeing a username there, I'm getting an error  
Jan 13 10:22:55 raspberrypi cron[368]: Error: bad username; while reading /etc/cron.d/pihole  
Jan 13 10:22:55 raspberrypi cron[368]: (_system_pihole) ERROR (Syntax error, this crontab file will be ignored)  
Jan 13 10:22:55 raspberrypi cron[368]: (CRON) INFO (Running @reboot jobs)

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 13, 2017, 10:55am UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/10 "2017-01-13T10:55:30Z")

</div>

Okay, so both of us have found something. It seems that I really have to do add the user there for Raspbian (that is not necessary under Ubuntu, where it defaults to `root`).

For debugging I changed the cron command to:

```auto
@reboot root /usr/sbin/logrotate -v /etc/.pihole/advanced/logrotate > /home/pi/logrotate 2>&1

```

That works because:

```auto
# cat /home/pi/logrotate
reading config file /etc/.pihole/advanced/logrotate

Handling 1 logs

rotating pattern: /var/log/pihole.log after 1 days (5 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/pihole.log
  log does not need rotating

```

From `syslog`:

```auto
Jan 13 11:53:45 raspberrypi CRON[495]: (root) CMD (/usr/sbin/logrotate -v /etc/.pihole/advanced/logrotate > /home/pi/logrotate 2>&1)

```

---

<div class="post-metadata">

**Author:** ![jpgpi250](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/jpgpi250/32/286_2.png) [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)\
**Post date:** [January 13, 2017, 11:01am UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/11 "2017-01-13T11:01:52Z")

</div>

the problem with linux flavors, there's always something different...

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 16, 2017, 8:14pm UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/12 "2017-01-16T20:14:46Z")

</div>

> [@jpgpi250](#):
>
> Adding missingok doesn't solve this, it just suppresses the error message, witch I would not advise, an error message indicates something is wrong.

I missed that one. No, having no log at all is perfectly fine since the users can always decide to completely disable logging (even during the first install) - a log file will never be present and forcing a reboot of `dnsmasq` is this case is not a good thing to do.

`logrotate` is now implemented and will become available in the next release.

---

<div class="post-metadata">

**Author:** ![Mcat12](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/mcat12/32/20_2.png) [@Mcat12](https://discourse.pi-hole.net/u/Mcat12)\
**Post date:** [January 29, 2017, 12:28am UTC](https://discourse.pi-hole.net/t/implement-logrotation-for-var-log-pihole-log/1266/13 "2017-01-29T00:28:49Z")

</div>