Core
Version is v5.18.3-549-g03932e8c (Latest: null)
Branch is development
Hash is 03932e8c (Latest: 03932e8c)
Web
Version is v5.21-1027-g96f9ee6a (Latest: null)
Branch is development
Hash is 96f9ee6a (Latest: 96f9ee6a)
FTL
Version is vDev-8478079 (Latest: null)
Branch is development
Hash is 84780798 (Latest: 84780798)
Your debug log shows that both NTP and DNS do time out at times:
*** [ DIAGNOSING ]: contents of /var/log/pihole
-rw-r----- 1 pihole pihole 5.9K Nov 28 09:29 /var/log/pihole/FTL.log
-----head of FTL.log------
2024-11-28 03:45:03.430 CET [25437/T25450] ERROR: Cannot receive UDP DNS reply: Timeout - no response from upstream DNS server
2024-11-28 05:28:35.590 CET [25437/T25447] WARNING: Could not recv() in reply() (/app/src/ntp/client.c:223): Resource temporarily unavailable
2024-11-28 05:28:35.590 CET [25437/T25447] ERROR: Failed to receive data from NTP server europe.pool.ntp.org: Timeout
-----tail of FTL.log------
2024-11-28 08:29:29.478 CET [25437/T25447] WARNING: Could not recv() in reply() (/app/src/ntp/client.c:223): Resource temporarily unavailable
2024-11-28 08:29:29.478 CET [25437/T25447] ERROR: Failed to receive data from NTP server europe.pool.ntp.org: Timeout
2024-11-28 08:29:31.588 CET [25437/T25447] INFO: Received 7/8 valid NTP replies from europe.pool.ntp.org
2024-11-28 08:29:31.588 CET [25437/T25447] INFO: Time offset: 1.505859e+01 ms (excluded 1 outliers)
2024-11-28 08:29:31.588 CET [25437/T25447] INFO: Round-trip delay: 2.723217e+01 ms (excluded 1 outliers)
2024-11-28 08:40:50.438 CET [25437/T25450] ERROR: Cannot receive UDP DNS reply: Timeout - no response from upstream DNS server
This could suggest that upstream servers are unresponsive at times (e.g. when under high load), or perhaps a network connectivity issue.
Your debug log shows your RPi hosting Pi-hole is connected via ethernet.
How does your router connect to your ISP?
Oh, that is very interesting indeed, as this is already a chafed ntp server I added because I got the warning with the default. The Router is DSL cable connected and I have very little issues with the connection otherwise.
Could this be an unbound issue?
Again, as this dient impact the pi-hole working as such abs the issue is apparently clearly in my network, while I appreciate the feedback, please don't hesitate "closing the ticket"
Most Internet upstream connections show sporadic failures on the order of very few seconds throughout the entire day. You typically don't notice this because you may not be browsing in that very second of the hour. Or it is just a small delay but the page still loads a fraction of time later due to the inherent retry nature of TCP (and also DNS). Or, this warning may has gone up within the few seconds in 24 hours where your DSL/cable model is renegotiating on the wire.
I am starting to wonder if we should change how/when this warning is shown. Currently, FTL is hyper-sensitive about this. If ever only one package is not received, it prints an error to the log. More and more, I get the feeling that we should only warn if no packets at all have been received across several sync intervals (= after several hours).
Hi @DL6ER , is it possible to also output the IP in the logs and below output?
$ pihole-FTL ntp europe.pool.ntp.org
Using NTP server: europe.pool.ntp.org
........
Received 8/8 valid NTP replies from europe.pool.ntp.org
Time offset: 1.834631e-01 ms (excluded 0 outliers)
Round-trip delay: 4.282463e+01 ms (excluded 0 outliers)
Please check out the new custom FTL branch tweak/ntp_errors using
sudo pihole checkout ftl tweak/ntp_errors
It has three changes regarding the NTP implementation:
If there was a timeout, we log the used IP address in addition to the host name. This may be useful to determine if only specific servers cause timeouts (as suggested by @deHakkelaar)
Do not issue double warnings when receiving NTP replies fails, e.g. skip the first of these two:
2024-11-28 05:28:35.590 CET [25437/T25447] WARNING: Could not recv() in reply() (/app/src/ntp/client.c:223): Resource temporarily unavailable
2024-11-28 05:28:35.590 CET [25437/T25447] ERROR: Failed to receive data from NTP server europe.pool.ntp.org: Timeout
In case the upstream server sends a "kiss code", we interpret and show it (e.g. "access denied" when authentication is required or "rate exceeded" when querying too often). Before, we simply logged
$ pihole-FTL ntp europe.pool.ntp.org
Using NTP server: europe.pool.ntp.org
Config file /etc/pihole/pihole.toml not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.1 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.2 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.3 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.4 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.5 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.6 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.7 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.8 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.9 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.10 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.11 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.12 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.13 not available (r): Permission denied
Config backup file /etc/pihole/config_backups/pihole.toml.14 not available (r): Permission denied
No config file nor backup available, using defaults
........
Received 8/8 valid NTP replies from europe.pool.ntp.org
Time offset: 1.697123e+00 ms (excluded 0 outliers)
Round-trip delay: 2.340746e+01 ms (excluded 0 outliers)
$ sudo pihole-FTL ntp gateway.fe2.apple-dns.net
Using NTP server: gateway.fe2.apple-dns.net
Failed to receive data from NTP server gateway.fe2.apple-dns.net (17.248.236.66): Timeout
Failed to receive data from NTP server gateway.fe2.apple-dns.net (17.248.236.66): Timeout
Failed to receive data from NTP server gateway.fe2.apple-dns.net (17.248.236.66): Timeout
Failed to receive data from NTP server gateway.fe2.apple-dns.net (17.248.236.66): Timeout
Failed to receive data from NTP server gateway.fe2.apple-dns.net (17.248.236.66): Timeout
Failed to receive data from NTP server gateway.fe2.apple-dns.net (17.248.236.66): Timeout
Failed to receive data from NTP server gateway.fe2.apple-dns.net (17.248.236.66): Timeout
Failed to receive data from NTP server gateway.fe2.apple-dns.net (17.248.236.66): Timeout
Warning NTP client: No valid NTP replies received, check server and network connectivity
Vs before:
$ sudo pihole-FTL ntp gateway.fe2.apple-dns.net
Using NTP server: gateway.fe2.apple-dns.net
Could not recv() in reply() (/app/src/ntp/client.c:223): Resource temporarily unavailable
Failed to receive data from NTP server gateway.fe2.apple-dns.net: Timeout
Could not recv() in reply() (/app/src/ntp/client.c:223): Resource temporarily unavailable
Failed to receive data from NTP server gateway.fe2.apple-dns.net: Timeout
Could not recv() in reply() (/app/src/ntp/client.c:223): Resource temporarily unavailable
Failed to receive data from NTP server gateway.fe2.apple-dns.net: Timeout
This shouldn't be, your pihole.toml should be owned by pihole, not root:
$ ls -lh /etc/pihole/pihole.toml
-rw-rw---- 1 pihole pihole 51K Nov 16 20:48 /etc/pihole/pihole.toml
Did you do anything to this file? Even if pihole-FTL is executed as root, it should detect this and chown the TOML file at the end to user pihole. I did just test this and it seems to work as it should. Does this happen on a native install or inside a container?
No I have not touched that file since I installed the beta on a fresh dedicated VM about a year ago.
I only upgraded Pi-hole a couple of times and also just before I run the checkout ftl tweak/ntp_errors.
If I need to change any, I use pihole-FTL --config.
$ pihole version
Core
Version is v5.18.3-549-g03932e8c (Latest: null)
Branch is development
Hash is 03932e8c (Latest: 03932e8c)
Web
Version is v5.21-1031-g9f9c26c5 (Latest: null)
Branch is development
Hash is 9f9c26c5 (Latest: 9f9c26c5)
FTL
Version is vDev-7c6e315 (Latest: null)
Branch is tweak/ntp_errors
Hash is 7c6e315b (Latest: 7c6e315b)
dehakkelaar@ph6c:~$ pihole-FTL ntp europe.pool.ntp.org
Using NTP server: europe.pool.ntp.org
........
Received 8/8 valid NTP replies from europe.pool.ntp.org
Time offset: -2.892047e+00 ms (excluded 0 outliers)
Round-trip delay: 5.921179e+01 ms (excluded 0 outliers)
dehakkelaar@ph6c:~$ pihole version
Core
Version is v5.18.3-505-g65806a90 (Latest: null)
Branch is development
Hash is 65806a90 (Latest: 03932e8c)
Web
Version is v5.21-984-gae728a16 (Latest: null)
Branch is development
Hash is ae728a16 (Latest: 9f9c26c5)
FTL
Version is vDev-d749dff (Latest: null)
Branch is development
Hash is d749dffb (Latest: 84780798)
Ah, sure. You dud run the job as user dehakkelaar != pihole and then 660 won't give you access. This is probably correct, though, so users cannot read the password hash, etc. (even if they couldn't do anythint useful with it).
$ tee -a ~/.bash_aliases <<< "alias ftlntp='sudo -u pihole pihole-FTL ntp'"
alias ftlntp='sudo -u pihole pihole-FTL ntp'
$ . ~/.bash_aliases
$
$ ftlntp localhost
Using NTP server: localhost
........
Received 8/8 valid NTP replies from localhost
Time offset: -9.059906e-03 ms (excluded 2 outliers)
Round-trip delay: 5.094210e-01 ms (excluded 2 outliers)
I've been trying out this new branch for a few days and I looks like the memory consumption climbs over time. I've seen as high as 80% of my Pi5's 8GB, not all from Pi-hole of course, but most of it.
When I run htop I see a whole bunch of Pi-hole processes and as time goes on more and more show up. The more appear, the more memory is consumed. These 'extra' processes don't appear to have used any CPU cycles either.
This doesn't appear to happen on the development branch and the memory usage is definitely lower there too.
