On Arch Linux, the below systemd service is operative for pihole-FTL. What I am noticing running with DBINTERVAL=180 is that I lose some data when I reboot the container running pihole. I think that is because, the way to get pihole.db to get updated from the cache, is to restart it first and then stop it, and our systemd service does not do that currently.
Current /usr/lib/systemd/system/pihole-FTL.service:
I tried adding these two lines thinking that the first would cause the database to get updated and the second would stop the pihole-FTL but when I check the time stamp on pihole.db it is not updated. Any thoughts are welcomed.
The long-term query database was the first database that was added to the Pi-hole project. We update this database periodically and on the exit of FTL DNS (triggered e.g. by a service pihole-FTL restart ).
Yes, from the package you called out which I co-maintain. I would like to modify the service therein but to do so, I need to know how the native pihole-FTL triggers the a database write from cache. I am not asking for any unofficial support.
Yes, that is what I mean. I tried various arguments including kill -HUP but nothing works if the end point is to see that the date/time stamp on /etc/pihole/pihole.db changes.
To simplify things, I took the system service out of the mix and simple ran the pihole-FTL like this:
% sudo -u pihole /usr/bin/pihole-FTL -f
[2024-07-14 08:12:09.341 3449M] Using log file /run/log/pihole-ftl/pihole-FTL.log
[2024-07-14 08:12:09.341 3449M] ########## FTL started on pihole! ##########
[2024-07-14 08:12:09.341 3449M] FTL branch: master
...
[2024-07-14 08:12:09.372 3449M] Listening on port 4711 for incoming IPv6 telnet connections
[2024-07-14 08:12:09.372 3449M] Listening on port 4711 for incoming socket telnet connections
[2024-07-14 08:12:09.373 3449M] INFO: FTL is running as user pihole (UID 973)
I tried sending the signal from another terminal but found that no output changed from pihole-FTL nor did the date-time stamp change on /etc/pihole/pihole.db ... what am I missing?
# kill -HUP $(pidof pihole-FTL)
# echo $?
0
If I drop the -HUP argument, and simply run kill the daemon does get terminated as expected but again, the date-time stamp does not change.
small summary (obviously under archlinux):
FTL correctly responds and flushes the database under HUP, QUIT, and TERM signals in a terminal context.
In a systemd service context the only way to get the flush is:
FTL does a final database export on exit (TERM) which should be what systemd is using by default, no? There is no reason to issue a pre-stop signal of any kind - just give it enough time to shut down on its own. When you send SIGTERM to pihole-FTL, the log file /var/log/pihole/FTL.log should be pretty explicit about this, e.g.
[...]
2024-07-14 15:12:06.385 CEST [1407537M] INFO: Asked to terminate by "/sbin/init" (PID 1, user root UID 0)
2024-07-14 15:12:06.385 CEST [1407537M] INFO: Shutting down... // exit code 0 // jmpret 0
2024-07-14 15:12:06.713 CEST [1407537M] INFO: Finished final database update
2024-07-14 15:12:06.713 CEST [1407537M] INFO: Waiting for threads to join
2024-07-14 15:12:06.713 CEST [1407537M] INFO: Thread database (0) is idle, terminating it.
2024-07-14 15:12:06.714 CEST [1407537M] INFO: Thread housekeeper (1) is idle, terminating it.
2024-07-14 15:12:06.714 CEST [1407537M] INFO: Thread dns-client (2) is idle, terminating it.
2024-07-14 15:12:06.714 CEST [1407537M] INFO: Thread timer (3) is idle, terminating it.
2024-07-14 15:12:06.714 CEST [1407537M] INFO: Thread ntp-client (4) is idle, terminating it.
2024-07-14 15:12:06.714 CEST [1407537M] INFO: All threads joined
2024-07-14 15:12:06.718 CEST [1407537M] INFO: Stored 0 API sessions in the database
2024-07-14 15:12:07.756 CEST [1407537M] INFO: ########## FTL terminated after 34m 59s (code 0)! ##########
[...]
HUP is not dumping to the database - it never has, not sure where you got this from. HUP is used by the embedded dnsmasq to flush the DNS cache, re-read parts of the config files. We use it after pihole -g as there may be domains in the DNS cache which are now to be blocked (or, not to be blocked any longer).
% sudo -u pihole /usr/bin/pihole-FTL -f
[2024-07-14 14:00:26.222 4682M] Using log file /run/log/pihole-ftl/pihole-FTL.log
[2024-07-14 14:00:26.222 4682M] ########## FTL started on pihole! ##########
[2024-07-14 14:00:26.222 4682M] FTL branch: master
[2024-07-14 14:00:26.222 4682M] FTL version: v5.25.2
...
[2024-07-14 14:00:26.253 4682M] Listening on port 4711 for incoming IPv6 telnet connections
[2024-07-14 14:00:26.253 4682M] Listening on port 4711 for incoming socket telnet connections
[2024-07-14 14:00:26.253 4682M] INFO: FTL is running as user pihole (UID 973)
[1] 4679 terminated sudo -u pihole /usr/bin/pihole-FTL -f
In another terminal I simply ran:
% sudo kill $(cat /run/pihole-ftl/pihole-FTL.pid)
This result is consistent if I start/stop it with systemd. If I tail -f /run/log/pihole-ftl/pihole-FTL.log and issue systemctl stop pihole-FTL no lines are written to the log and the process dies.
even my FTL doesn't write anything you pasted!!!
no database update, no join and sync on threads, nothing, it just dies....
is maybe something wrong with our compilation? Like multi threading inactive or something like that?
does that make sense what I am saying?
Note that what I pasted actually comes from a v6 Pi-hole, output from a v5.x Pi-hole might look slightly different but conceptually it will be the same, e.g.
I can only repeat that something is very wrong here. From all your input, it seems like pihole-FTL never receives the SIGTERM (but instead SIGKILL) or if SIGTERM is followed by SIGKILL so closely that it doesn't even have a chance to log a message about planing to shut down.
This may be a fundamental Arch Linux limitation, but this is just a hypothesis (and not even a very good one) - the only thing we know is that it appears you are sending SIGTERM to pihole-FTL but SIGKILL is effective in the end.
Maybe you can repeat this with specifying SIGTERM explicitly. I am really picking at straws here. Do you know if Arch Linux does the same to other services which are not granted any time for ordinary shutdowns?
edit I checked again and see that the most immediate feedback will be logged to /var/log/pihole/pihole.log:
After starting the systemd service, I tried both of these methods of killing pihole-FTL but neither resulted in any additional output to the log as you showed:
Unfortunately it's seems not a "simple" systemd service problem: at least in my system, the behavior is also the same by starting FTL and sending the TERM signal “by hand”