Long-term data not stored it seems

[ChurchOfNoise]

Expected Behaviour:

Pi-Hole stores data over the defined standard period (I am running on the latest version of the dev branch so should be 1 month of data IIRC)

Actual Behaviour:

Bizarrely, Pi-Hole has stopped saving data in the long-term log since Oct 2nd:

Trying to look at the tail of pihole-FTL.log (from the web interface) shows no data, the tail of pihole.log is populated though.

FYI: I'm running the latest version of Pi-Hole on the dev branch + Unbound + Log2Ram

Debug Token:

https://tricorder.pi-hole.net/yn9k6iox5c
(don't worry that 8.8.8.8 can't resolve the test, I blocked Google IPV4 DNS servers at my router)

[Bucking_Horn]

Let's see how often data gets written to the db on your machine by running:

grep -ni "DBINTERVAL" /etc/pihole/pihole-FTL.conf

And let's verify whether that would be a display glitch or that data is indeed missing from your db.
The following SQL statement will return the number of domains Pi-hole was asked to resolve since October 2nd:

sqlite3 /etc/pihole/pihole-FTL.db \
"SELECT count(domain) FROM queries \
WHERE(timestamp BETWEEN strftime('%s','2020-10-02') AND strftime('%s','2020-10-10'));"

Please adjust dates as required.

You may also want to check for drive space on your Pi-hole machine:

df -h

[ChurchOfNoise]

So, another update: after changing a couple of things in 'Local DNS' the count started from 0 again (as did the data Unbound captures) and now all seems to work.

I did use the commands you provided:

@Bucking_Horn

Let's see how often data gets written to the db on your machine by running:

So, doing that:

• Between Oct 2 and today: 29900
• Between Oct 2 and Oct 8: 14142

And

@Bucking_Horn

You may also want to check for drive space on your Pi-hole machine:

df -h

Gives:

Filesystem      Size  Used Avail Use% Mounted on
/dev/root        30G  1.7G   27G   6% /
devtmpfs        208M     0  208M   0% /dev
tmpfs           240M  1.7M  239M   1% /dev/shm
tmpfs           240M  3.4M  237M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           240M     0  240M   0% /sys/fs/cgroup
/dev/mmcblk0p1  253M   54M  199M  22% /boot
log2ram         100M   46M   55M  46% /var/log
tmpfs            48M     0   48M   0% /run/user/1000
tmpfs            48M     0   48M   0% /run/user/999

[Bucking_Horn]

Your df result is ok, so it's not an issue of available drive space.

Assuming that searching for DBINTERVAL returned nothing, you'd be using the default, i.e. writing to db every minute.

But while your counts show some activity, it would seem your db has recorded about the same amount of queries during a week you'd normally see in a day.

Just to be sure: Would that fit your actual expected usage, e.g. having been away for a while during that time?

[ChurchOfNoise]

Indeed, DBINTERVAL returned nothing.

Agreed on your observation on the activity. I was however not out so the sudden lack of recorded activity seems like the result of an issue somewhere...
(FYI, I check drive space every now and then to make sure my log2ram parameters are ok).

[ampfinger]

Also had the same issue running latest dev branch.

This PR helped me and fixed it:

Keep FTL database open

[ChurchOfNoise]

Reverted to 'main' development branch last night after merge of this PR, and the problem seems to have somehow reappeared: some data has gone missing. The data was there before I added another device in the 'Local DNS' list, afterwards it just vanished. Data is also gone from long term db (checked on today's data)

Have run pihole -up now and it seems there is another update for FTL still, but if I look at Github, this is probably a merge of some other functionality...

debug token: https://tricorder.pi-hole.net/0nftxc08k5