Please follow the below template, it will help us to help you!
Expected Behaviour:
DNS queries should get logged for access in the UI
Actual Behaviour:
Overall the system is working as intended. However I encountered an issue with large amounts of log data causing a problem.
I had a LAN client really hammer my DNS today - 1.5 million queries in 24hrs - and it generated almost 2gb of logs and filled up /var.
It appears that every query is recored twice, once in pihole.log and once pihole-FTL.log? Why is this? Which one is used by the web UI and how do I reduce verbosity on the other one?
You may view pihole.log as dnsmasq's standard log file, and pihole-FTL.log as the file containing additional log output by pihole-FTL.
Depending on your configuration, there may be an overlap, but generally, they don't contain the exact same information twice.
Since I recall us hunting down issues together in the past, you may want to check your /etc/pihole/pihole-FTL.conf for any DEBUG_ options and remove them if you no longer need them.
I additon, you may decide to impose your own limits on Pi-hole's log rotation scheme by editing /etc/pihole/logrotate.
Also, note that while Pi-hole's log files will usually expire and vanish after a few days, query information in its long term database will persist for much longer (controlled by pihole-FTL.conf's MAXDBDAYS).
So above all, I'd like to recommend to single out the offending client in order to find and potentially eliminate the cause of those excessive amount of DNS requests.
As additional hint, I can suggest to disable Pi-hole's query logging completely if you don't need it (you don't need it = nothing has to be debugged - these logs are vital for support requests):
pihole logging off
(note that this will also flush the existing logs)
Yep, we've worked together before - lol. Good to hear from you again Yes, there were two debug_ lines in the conf file and I've turned that off. Thanks!
I'll tinker with the logrotate settings as well, because it's also not compressing the XYZ.log.### rolled logs.
Does disabling logging still show details about top requested records, clients, etc? That kind of data has been exceedingly helpful in diagnosing problematic clients in my house (such as that one laptop that suddenly went berserk with DNS requests when it dropped off the work VPN it was connected to).
If all of that would remain intact, I've got no problem disabling it unless needed for support requests.
Yes. This information is stored in the long term database at /etc/pihole/pihole-FTL.db.
You can generally control the size of this database through the MAXDBDAYS parameter. The default is 365 - one year of data in the database.
The dnsmasq log rotates nightly and only 6 days are kept (and all but two are uncompressed). Any large file size surges from lots of queries are gone in six days:
ls -lha /var/log/pihole.log*
-rw-r--r-- 1 pihole pihole 1.8M Dec 19 11:03 /var/log/pihole.log
-rw-r--r-- 1 pihole pihole 5.0M Dec 19 00:00 /var/log/pihole.log.1
-rw-r--r-- 1 pihole pihole 248K Dec 18 00:00 /var/log/pihole.log.2.gz
-rw-r--r-- 1 pihole pihole 264K Dec 17 00:00 /var/log/pihole.log.3.gz
-rw-r--r-- 1 pihole pihole 234K Dec 16 00:00 /var/log/pihole.log.4.gz
-rw-r--r-- 1 pihole pihole 273K Dec 15 00:00 /var/log/pihole.log.5.gz
Yes, there were two debug_ lines in the conf file and I've turned that off. Thanks!