No query log update anymore and no top clients (all and blocked)

General Beta 6.0
1

Hi, I just updated to following versions:

  • Core vDev (development-v6, cc8f9fe0)
  • FTL vDev (development-v6, ce4407b5)
  • Web interface vDev (development-v6, 1dcf90bd)

and have no query list updates anymore and also the top clients aren't shown anymore at dashboard.

Numbers are counted and looks like blocking and resolving works fine

2

Flush log didn't change anything
in dashboard I see 2230 total and 1446 blocked but in query log are only 52

3

Please upload a debug log and post just the token URL that is generated after the log is uploaded by running the following command from the Pi-hole host terminal:

pihole -d
4

https://tricorder.pi-hole.net/KndwLFyG/

5

btw. If I go back to a older version (I use proxmox and used a backup from 21st of January) all works fine:

  • Core vDev (development-v6, e1c41fe4)
  • FTL vDev (development-v6, 3341f0e4)
  • Web interface vDev (development-v6, de5a4b0d)
7

Is your database on the same host as the VM? Which filesystem are you using?

8

It's a Debian 12 LXC container and PI-hole is all and the only whats running there, No access to other LXC's. Also standard ext4 filesystem and as I told it's running after backup older version I mentioned in prior task fine again.

9

Any idea what the reason can be?

10

Your debug log shows two issues - while the first one may or not be the cause for the second one:

  1. Your log contains a warning that the database was not closed properly. Maybe the container was restarted abruptly or something else happened that didn't allow the pihole-FTL process to handle the database correctly.
  2. As a consequence of no. 1 above (or completely independent, we don't know yet), your database got locked. This prevented pihole-FTL from reading your gravity database resulting in many warnings in your /var/log/pihole/FTL.log. And - apparently - down the line also prevented it from accessing any query data at all.

Please try starting from a fresh database to eliminate a database corruption as cause using:

sudo service pihole-FTL stop
sudo mv /etc/pihole/pihole-FTL.db /etc/pihole/pihole-FTL_broken.db
sudo rm /etc/pihole/pihole-FTL.db*
sudo service pihole-FTL start

inside the container.

Also of interesting is the output of:

pihole-FTL sqlite3 /etc/pihole/pihole-FTL_broken.db "PRAGMA integrity_check;"

(this command may take some time to finish, there is no progress reporting)

11

No change at all after doing the recomended steps and:
pihole-FTL sqlite3 /etc/pihole/pihole-FTL_broken.db "PRAGMA integrity_check;"
ok

12

Back to prior version and all fine again. If I can help further investigation, let me know

13

Thank for you the offer and sorry for the long delay in responses... First, we should investigate if this is due to the WAL journal or due to some other problem (which will be harder to trace).

Please run

sudo pihole checkout ftl new/useWAL
sudo pihole-FTL --config database.useWAL false

to get an experimental branch which allows controlling the WAL mode and, subsequently, turning it off.

FTL from this branch should be available in a few minutes. Until then, you'd get a harmless error during the checkout and nothing happens.

edit Wait, I just had another look into your debug log and see that you have non-empty setting for webserver.api.excludeClients and webserver.api.excludeDomains. We merged a feature last week that extended these options to also be effective for the Query Log:
https://github.com/pi-hole/FTL/pull/1611

Could you please check if editing these (e.g. via the web interface's settings -> all settings page) resolves the issue you are seeing? I see you are using a limited set but - maybe just maybe - if all your queries are either localhost or fritz.box, then this would easily explain what is happening here.

14

so, removing all entries in webserver.api.excludeClients and webserver.api.excludeDomains fixed the behavior,

But how can I use this feature in future? And also how will you handle this, if you will once do a upgrade for all users from V5 to V6 which are using this feature in V5?

15

There is no issue with using these settings at all. We still don't really know why the settings have been an issue for you. Is that that

?

16

exclude clients:
localhost
fritz.box
iobroker.fritz.box
SKY-IPTV-WZ.fritz.box
SKY-IPTV-GZ.fritz.box
DiskStation.fritz.box
pihole-6.fritz.box

exclude domains:
www.example.com
www.example.net
www.example.org
ipinfo.io
cloudsync-tw.synology.com
checkip.synology.com
fritz.box
www.gstatic.com
connectivitycheck.gstatic.com
wpad.fritz.box
connectivity.sky.com

17

Yes, I have seen these in your debug log. My point is:

When you have localhost and fritz.box as clients to be excluded and all your queries either come from localhost or fritz.box (because you are using the DNS server of your FritzBox instead of distributing the Pi-hole as DNS server), then everything is behaving as expected.

The only bit that was not expected is that the functionality of these settings was extended to the Query Log.

18

ok, but what's the opions then? (Pi-hole as dns doesn't work for me )

19

Why?

20

forget it - I use Pi-hole as DNS (I mean as DHCP doesn't work)

21

Where do you see, that I'm not using Pi-hole as DNS? fritz-box as DHCP sends only the IP of Pi-Hole to the clients (v4 and v6) and also use it by itself