I'm using development branch with FTL.
After flushing logs in settings, queries are still exists. But after flushing two times, it will be successful. Any other users experiencing this issue?
This is to be expected. In order to be able to show the rolling 24h window, FTL analyzes *both the current log as also the first rotated file (/var/log/pihole.log.1). When you flush, the data from pihole.log is moved into pihole.log.1 and then still read by FTL. Only if you flush twice, you can move the data entirely out-of-view of FTL.
Do you have a suggestion on how to change that?
Note: If you disable the rolling 24h windows (displaying data only starting from midnight, as we have it in the current master version), then flushing once will, again, be sufficient.
What about providing two buttons to flush logs? Or an option in query log page to show logs from 24hr window or not?
How to label the buttons? Something like "flush recent 24h" + "flush recent 48h" is not correct if the log is younger. "Flush Once" + "Flush Twice" will most likely not understood by the user.
At some point, we have to add the Settings page elements for FTL, I know, but that is still to be done (and they still change from time to time that's why I didn't did that already).
Flush all - cleans everything.
Flush - moves log file into /var/log/pihole.log.1
But flush all is ambiguous. Assume the user has decided to only use the display from midnight. Then flush will already do what he expected.
Having said that, why shouldn't we remove flushing altogether? I don't see a situation where it would be necessary, anymore. We have tested scenarios with several million queries on Raspberries and even several hundreds of millions of queries on hardware with more memory available (e.g. ~ 130 mio queries on a VM with 4 GB of RAM + 4GB of swap).
The only common situation I can see it needed for is privacy. So perhaps making the flush button fully flush both day's queries would be sufficient.
1 Like
See
This feature request (clicking the button moves all data out-of-sight of FTL) has been implemented and will be included in the next release.
1 Like
I am also use the FTL dev version for testing.
And I have noticed that the queries counter will not set back to 0 after midnight or after 24h, although they labeled to show the queries of today.
@DL6ER, I most commonly flush my logs after any reboots of the router or the pi-hole. As I use DNScrypt on the pi-hole it can talk a little while to bring everything up, and I have to do it in order. During this process, I will get many unwanted entries in the hosts list of my logs, which I do not want to look at. Only once everything is back up and running properly, then I flush for a clean start, and work off that.
TLDR; I flush to reset my connected hosts lists, not necessarily how many blocked or used DNS lookups.
Okay, thanks for your explanation!
1 Like
I find that logs are still available in Long Term Data | Query Logs even after flushing (and rebooting the system) twice. I've also adjusted privacy settings to only log blocked queries, yet permitted entries still show up in the Query Log under Long Term Data. Even after manually deleting all pihole logs in var/log/ and rebooting, I can still pull up the last 7 days of query logs via Long Term Data! How can I get rid of this Long Term Data?
There is a fix in the next release.
The fix above executes the following single-line command. Maybe you can use it as manual workaround.
Delete most recent 24 hours from FTL's database, leave even older data intact (don't wipe out all history)
sqlite3 "/etc/pihole/pihole-FTL.db" "DELETE FROM queries WHERE timestamp >= strftime('%s','now')-86400; select changes() from queries limit 1"
From pihole -f: Flush database by DL6ER · Pull Request #1929 · pi-hole/pi-hole · GitHub, "# Delete most recent 24 hours from FTL's database, leave even older data intact (don't wipe out all history)".
Why leave older data intact? This seems counter-intuitive to the meaning of the word "flush", and is probably not what people expect when they click on the button. Why not flush everything? At the very least, it seems like there should be some warning that data older than 24 hours is being left intact.
Update: read some of the previous posts and see that this has already been mentioned.
For me, privacy is the driver. I don't want or need a record of every site I visit sitting around. I might want to enable logging temporarily for troubleshooting purposes, but then I'd like to be able to disable logging and flush everything.
What about 'MAXDBDAYS=...' in /etc/pihole/pihole-FTL.conf
You can either use the suggestion of @mibere or stop pihole-FTL, manually remove the database file and restart pihole-FTL.
The flushing does only remove the most recent 24 hours as this is what we do since a longer time already, the only difference is that we now load the history from the database instead of the log files. In this sense, the deletion of 24h of data is just enough to remove all data from the dashboard.
We introduce also the concept of privacy levels, which rather seems to be what you are looking for. Information concerning this is available a bit scattered around on this forum. I'm planing to implement settings as well as a proper description of the principle as soon as I can.
Here is a comment about the privacy levels
https://discourse.pi-hole.net/t/help-us-beta-test-ftldns/8224/42?u=mibere
1 Like
See also
I wrote that some time ago and have to recheck this description, but it should come at least close