Logs are not flushing correctly

See

This feature request (clicking the button moves all data out-of-sight of FTL) has been implemented and will be included in the next release.

1 Like

I am also use the FTL dev version for testing.
And I have noticed that the queries counter will not set back to 0 after midnight or after 24h, although they labeled to show the queries of today.

@DL6ER, I most commonly flush my logs after any reboots of the router or the pi-hole. As I use DNScrypt on the pi-hole it can talk a little while to bring everything up, and I have to do it in order. During this process, I will get many unwanted entries in the hosts list of my logs, which I do not want to look at. Only once everything is back up and running properly, then I flush for a clean start, and work off that.

TLDR; I flush to reset my connected hosts lists, not necessarily how many blocked or used DNS lookups.

Okay, thanks for your explanation!

1 Like

I find that logs are still available in Long Term Data | Query Logs even after flushing (and rebooting the system) twice. I've also adjusted privacy settings to only log blocked queries, yet permitted entries still show up in the Query Log under Long Term Data. Even after manually deleting all pihole logs in var/log/ and rebooting, I can still pull up the last 7 days of query logs via Long Term Data! How can I get rid of this Long Term Data?

There is a fix in the next release.

The fix above executes the following single-line command. Maybe you can use it as manual workaround.

Delete most recent 24 hours from FTL's database, leave even older data intact (don't wipe out all history)

sqlite3 "/etc/pihole/pihole-FTL.db" "DELETE FROM queries WHERE timestamp >= strftime('%s','now')-86400; select changes() from queries limit 1"

From pihole -f: Flush database by DL6ER · Pull Request #1929 · pi-hole/pi-hole · GitHub, "# Delete most recent 24 hours from FTL's database, leave even older data intact (don't wipe out all history)".

Why leave older data intact? This seems counter-intuitive to the meaning of the word "flush", and is probably not what people expect when they click on the button. Why not flush everything? At the very least, it seems like there should be some warning that data older than 24 hours is being left intact.

Update: read some of the previous posts and see that this has already been mentioned.

For me, privacy is the driver. I don't want or need a record of every site I visit sitting around. I might want to enable logging temporarily for troubleshooting purposes, but then I'd like to be able to disable logging and flush everything.

What about 'MAXDBDAYS=...' in /etc/pihole/pihole-FTL.conf

See GitHub - pi-hole/FTL: The Pi-hole FTL engine

You can either use the suggestion of @mibere or stop pihole-FTL, manually remove the database file and restart pihole-FTL.

The flushing does only remove the most recent 24 hours as this is what we do since a longer time already, the only difference is that we now load the history from the database instead of the log files. In this sense, the deletion of 24h of data is just enough to remove all data from the dashboard.

We introduce also the concept of privacy levels, which rather seems to be what you are looking for. Information concerning this is available a bit scattered around on this forum. I'm planing to implement settings as well as a proper description of the principle as soon as I can.

Here is a comment about the privacy levels

https://discourse.pi-hole.net/t/help-us-beta-test-ftldns/8224/42?u=mibere

1 Like

See also

I wrote that some time ago and have to recheck this description, but it should come at least close

See this feature request - the privacy level are now configurable via the dashboard settings page and the CLI

1 Like

Thanks. That should work for now. I still don't see the point of only flushing the last 24 hours, and I think a lot of people might mistakenly think they are flushing everything when they click that button.

If the [Flush logs] button doesn't flush the complete log(s) there should be a notice about this in the "Confirmation required" dialog. For me the current text in the confirmation dialog means that the complete log(s) will be kicked.

I think we can open a survey on this platform to ask users how they want the flushing to work ("24 hrs", "all history", or "don't care") - what do you think and could you set up something like this @jacob.salmela?

Are we talking about flushing the log file or long term stats?

How would people in this thread like the log file and long-term stats to be flushed?

  • 24 hours
  • all history
  • I don't care

0 voters

Dear community, vote. It's in your own interest :slight_smile:

1 Like