Please follow the below template, it will help us to help you!
Expected Behaviour:
When viewing the pihole dashboard, the chart should reflect query counts from the log files.
Actual Behaviour:
The query charts show spikes and gaps for random sections of time, though looking at the logs directly shows queries occurring during the missing time. This occurs randomly, sometimes every few hours, sometimes once per 24 hours.
Restarting pihole-FTL resolves this and after a page refresh the missing sections of time are filled in, with the gaps disappearing and the spikes 'leveling' out.
It could be difficult to tackle this, as information about the dashboard is not available from the debug log: The dashboard is strictly in-memory - possibly prepopulated by the last 24 hours from the long term database at startup, but growing in-memory from recording DNS requests as they arrive.
One possible reason for the dashboard disagreeing with the logs or the long term query database would be time issues, e.g. where browsing machine has different time or time zone information than the Pi-hole host machine.
Your debug log also contains a hint that the time may not always be correct on your Pi-hole host machine:
*** [ DIAGNOSING ]: contents of /var/log
lrwxrwxrwx 1 root root 42 Feb 7 2020 /var/log/pihole-FTL.log -> /media/usbdrive/logs/pihole/pihole-FTL.log
-----tail of pihole-FTL.log------
[2022-06-28 09:30:00.016 28641M] WARN: Found database entries in the future (2022-06-28 09:35:00 (1656423300), last timestamp for importing: 2022-06-28 09:15:00 (1656422100)). Your over-time statistics may be incorrect (found in src/dnsmasq_interface.c:667)
Those kind of errors can happen on RPis, as they lack a battery backuped RTC.
When starting after a powerdown (especially without a previous clean shutdown), the time may be off until the RPi is able to sync with a time server.
You should verify whether your Pi-hole host machine time and time zone is configured correctly.
This really may have been happening for a long time, but I noticed it recently b/c I had replaced my router and thought it was having connectivity gaps.
I've gone through all the TZ and date settings, but everything seems to match in all of my systems. For now, I removed the pihole-ftl.db and ran pihole -r. This gives me a good starting point. Thank you for your help!
It may not have been clear from my earlier remark: Those database entries in the future would occur upon a Pi-hole restart, when system time would lag behind real time because the system hasn't synced with a time server yet.
As an example, consider a sudden power outage: Pi-hole logs a query at 09:35, then the system stops unexpectedly because of a power cut. Once power returns, the RPi boots up again and sets its time to 09:30 (e.g. from the last time recorded by fake-hwclock on an RPi). When Pi-hole starts up, it reads the latest entry (09:35) from its database to populate the dashboard, checks that against the system time (09:30) and reports the mismatch.
That could partially explain your observation, provided that your Pi-hole host machine would have been restarted around the times you observe the gaps.
I've been running on the same 3B+ since I first set up Pihole. It is possible that this has occurred the entire time to some degree, since it's only the recent spikes like in the screenshot that tipped me off.
I have restarted the system, and just pihole-ftl, more than once, so I'll have to compare results. I know that the issue does randomly appear as well over time when there are no restarts. Makes sense for it to appear as described though.
After removing pihoile.db and running pihole -r, the issue did seem to occur within the first hour or so.
I read through that PR and it's possible it's related, so I'm following it now. Thank you again!
Just as the RTC implications, those overTime inaccuracies would involve a restart.
If you did observe any Pi-hole restarts around the time of the gaps, would those have been initiated by you, or would they have happened unexpectedly, e.g. due to power failures or an insufficient PSU?
