DNS service not running - FTL Offline

I seem to have an issue with DNS service not running and FTL being offline (PiHole dashboard GUI shows this info). If I configure my clients through PiHole, I am not able to get through to any website (as DNS is not running).

Troubleshooting I have done:

  • After RPi reboot, all works fine. It stops running after 8-10-12 hours.
  • I have tried to implement a few of the fixes that were suggest here, none were helpful, so I reversed them.
  • The issue started either after I updated PiHole OR after I installed OpenVPN (WireGuard). OpenVPN is working like a charm, I can even use PiHole on my VPN (when the DNS is working). In the config process OpenVPN detected PiHole, and OpenVPN is installed in a PiHole friendly way.

I have the following services running on my RPi (in the order of install):

  • PiHole
  • Torrent Client - Transmission Web :9091
  • MiniDLNA - UPnP Server :8200
  • AnyDesk
  • VNC Viewer
  • SMB Share
  • Plex Server - Port: 32400
  • PiVPN - WireGuard: 51820

Token: https://tricorder.pi-hole.net/k6xj7uu0uy

The following seems to be the most obvious issue:
*** [ DIAGNOSING ]: Dashboard and block page
[✗] Block page X-Header: X-Header does not match or could not be retrieved.
HTTP/1.1 200 OK
Content-type: text/html; charset=UTF-8
Expires: Thu, 27 May 2021 14:29:01 GMT
Cache-Control: max-age=0
Date: Thu, 27 May 2021 14:29:01 GMT
Server: lighttpd/1.4.53

My hardware:

  • Raspberry Pi 4B 8GB
  • 16Gb SD Storage
  • 500Gb Mounted HDD
  • LAN connection to my router

Two words:
security nightmare :smiley:

lol, why?
Not is you take specific steps to secure the system?

You have so many attack vectors open to like for example possibly poison your DNS cache that affects all devices in your LAN.
Separation is the key.
Run services on different hosts to separate your DNS from the other services.
Or go the Docker way for good separation.

Aha, okay. I don't have the hardware to run multiple Dockers.
Obviously I can't have individual RPi for each server.

So what would your suggestion of breaking this down be:

  • PiHole
  • Torrent Client - Transmission Web (Local)
  • MiniDLNA - UPnP Server (Local)
  • AnyDesk
  • VNC Viewer (Local)
  • SMB Share (Local share)
  • Plex Server - Port: 32400
  • PiVPN - WireGuard: 51820

And what is your setup like if you have these servers?

I have a dedicated Raspi 1B for Pi-hole & Unbound.
This dedicated Pi has no VNC viewer or AnyDesk.
Only entrance to administer is via SSH.
The rest I run on my NAS either as installable addon or in Docker if I choose so.

1 Like

Thanks for the suggestion.

I'm happy with my current setup for now, and it would be good to solve the outstanding issue with the DNS not running.

1 Like

I recall the PiVPN installation to introduce conflicting elements into configuration files, breaking Pi-hole's DNS operation eventually.
I am unaware if that would still be the case or if those have been addressed in the meantime.
It's probably worth trying again without PiVPN.

The most obvious is that pihole-FTL is not running and there is no content in /var/log/pihole-FTL.log that would give a hint to some malfunction.

What happens if you manually restart pihole? Are there any errors in the mentioned log?

sudo service pihole-FTL start

How would one find/replace these files?

I didn't not wait around to fix this this time round and sudo reboot now 12 hours ago to get the service working for the clients.

Once the DNS and FTL stop running again, I will try to re-start it manually and check pihole -d and report back here.


But now, I am getting the symptoms of the Offline FTL and DNS not running error.
DNS and FTL are still running, but I am getting the following errors/symptoms now:

No GUI Log data:



This error appears in Pi-Hole Dashboard > Query Log (It is also common error I believe):

DataTables warning: table id=all-queries - Tried 100 times to connect to FTL server, but never got proper reply. Please check Port and logs!

Pi-hole will create them on startup.


How do you judge this? From the green status indicator?

If this is try, then only your web interface would be broken according to the DataTables warning

Please generate a new debug log now.

UPDATE:
This issue still persists after yesterday's reboot, although DNS and FTL is still running:

*** [ DIAGNOSING ]: Dashboard and block page
[✗] Block page X-Header: X-Header does not match or could not be retrieved.
HTTP/1.1 200 OK
Content-type: text/html; charset=UTF-8
Expires: Fri, 28 May 2021 07:26:38 GMT
Cache-Control: max-age=0
Date: Fri, 28 May 2021 07:26:38 GMT
Server: lighttpd/1.4.53

Yes, the green status and it is blocking ads. The pihole -d reports this:

*** [ DIAGNOSING ]: Name resolution (IPv4) using a random blocked domain and a known ad-serving domain
[✓] www.clixtrac.com is 0.0.0.0 via localhost (127.0.0.1)
[✓] www.clixtrac.com is 0.0.0.0 via Pi-hole (192.168.2.108)
[✓] doubleclick.com is 142.250.181.206 via a remote, public DNS server (8.8.8.8)

When the Dashboard status is red (DNS and FTL not running) I am not able to go through to any website, as I mentioned in the first post.

Here is afresh Token: https://tricorder.pi-hole.net/m16g3v657y

Thank you for your help!

Here is the problem:

   [2021-05-28 09:14:00.366 718/T722] Encountered error while trying to store queries in long-term database: database is locked
   [2021-05-28 09:14:00.391 718/T722] ERROR: SQL query "DELETE FROM network_addresses WHERE lastSeen < 1590653640;" failed: database is locked
   [2021-05-28 09:14:00.391 718/T722] SQLite3 message: API call with invalid database connection pointer (21)
   [2021-05-28 09:14:00.391 718/T722] SQLite3 message: misuse at line 165242 of [5d4c65779d] (21)
   [2021-05-28 09:14:00.391 718/T722] Error while trying to close database: bad parameter or other API misuse
   [2021-05-28 09:15:00.466 718/T722] Encountered error while trying to store queries in long-term database: database is locked
   [2021-05-28 09:15:00.492 718/T722] ERROR: SQL query "DELETE FROM network_addresses WHERE lastSeen < 1590653700;" failed: database is locked
   [2021-05-28 09:15:00.492 718/T722] SQLite3 message: API call with invalid database connection pointer (21)
   [2021-05-28 09:15:00.492 718/T722] SQLite3 message: misuse at line 165242 of [5d4c65779d] (21)
   [2021-05-28 09:15:00.493 718/T722] Error while trying to close database: bad parameter or other API misuse
   [2021-05-28 09:16:00.568 718/T722] Encountered error while trying to store queries in long-term database: database is locked
   [2021-05-28 09:16:00.594 718/T722] ERROR: SQL query "DELETE FROM network_addresses WHERE lastSeen < 1590653760;" failed: database is locked
   [2021-05-28 09:16:00.594 718/T722] SQLite3 message: API call with invalid database connection pointer (21)
   [2021-05-28 09:16:00.594 718/T722] SQLite3 message: misuse at line 165242 of [5d4c65779d] (21)
   [2021-05-28 09:16:00.594 718/T722] Error while trying to close database: bad parameter or other API misuse
   [2021-05-28 09:17:00.670 718/T722] Encountered error while trying to store queries in long-term database: database is locked
   [2021-05-28 09:17:00.694 718/T722] ERROR: SQL query "DELETE FROM network_addresses WHERE lastSeen < 1590653820;" failed: database is locked
   [2021-05-28 09:17:00.695 718/T722] SQLite3 message: API call with invalid database connection pointer (21)
   [2021-05-28 09:17:00.695 718/T722] SQLite3 message: misuse at line 165242 of [5d4c65779d] (21)
   [2021-05-28 09:17:00.695 718/T722] Error while trying to close database: bad parameter or other API misuse
   [2021-05-28 09:18:00.771 718/T722] Encountered error while trying to store queries in long-term database: database is locked
   [2021-05-28 09:18:00.797 718/T722] ERROR: SQL query "DELETE FROM network_addresses WHERE lastSeen < 1590653880;" failed: database is locked
   [2021-05-28 09:18:00.798 718/T722] SQLite3 message: API call with invalid database connection pointer (21)
   [2021-05-28 09:18:00.798 718/T722] SQLite3 message: misuse at line 165242 of [5d4c65779d] (21)
   [2021-05-28 09:18:00.798 718/T722] Error while trying to close database: bad parameter or other API misuse

Something is wrong with your long-term database. You can try to recover it:

sudo service pihole-FTL stop
pihole-FTL /etc/pihole/pihole-FTL.db ".recover" | pihole-FTL ~/pihole-FTL_recovered.db
sudo mv ~/pihole-FTL_recovered.db /etc/pihole/pihole-FTL.db
sudo service pihole-FTL start

It retuned this:

pi@beck:~ $ sudo service pihole-FTL stop
pi@beck:~ $ pihole-FTL /etc/pihole/pihole-FTL.db ".recover" | pihole-FTL ~/pihole-FTL_recovered.db
Error: unknown command or invalid arguments:  "recover". Enter ".help" for help

Fresh token: https://tricorder.pi-hole.net/xklphcq4pv

So what is next after this? I am getting error on the .recover attribute.

Oh sorry. This function was added only recently to Pi-hole and is not part of the master branch at the moment. You can a) checkout the development branch (which contains this feature) or b) just move the database (loosing all historical data) and let FTL create a new database.

For the former

pihole checkout ftl development
sudo service pihole-FTL stop
pihole-FTL /etc/pihole/pihole-FTL.db ".recover" | pihole-FTL ~/pihole-FTL_recovered.db
sudo mv ~/pihole-FTL_recovered.db /etc/pihole/pihole-FTL.db
sudo service pihole-FTL start
pihole checkout ftl master

For option b)

sudo service pihole-FTL stop
sudo mv /etc/pihole/pihole-FTL.db /etc/pihole/pihole-FTL_old.db
sudo service pihole-FTL start
1 Like

Thank you, I have renamed the .db file now. It seems to be working fine for now, let's see if this resolves the problem of DNS stop running after a few hours.

Although, it still shows this error in the pihole -d. Or is there some other solution to "Block page X-Header" error?

*** [ DIAGNOSING ]: Dashboard and block page
[✗] Block page X-Header: X-Header does not match or could not be retrieved.
HTTP/1.1 200 OK
Content-type: text/html; charset=UTF-8
Expires: Sat, 29 May 2021 16:28:33 GMT
Cache-Control: max-age=0
Date: Sat, 29 May 2021 16:28:33 GMT
Server: lighttpd/1.4.53

Here is fresh token: https://tricorder.pi-hole.net/qgy2p2cs0c

You are not using a blocking mode that produces a block page, so this is not a problem. You can ignore this message.

Unrelated note - you may want to revisit your selection of upstream DNS servers. There is no reason to have them all selected.

*** [ DIAGNOSING ]: Setup variables
    PIHOLE_DNS_3=208.67.222.222
    PIHOLE_DNS_4=208.67.220.220
    PIHOLE_DNS_5=4.2.2.1
    PIHOLE_DNS_6=4.2.2.2
    PIHOLE_DNS_7=8.26.56.26
    PIHOLE_DNS_8=8.20.247.20
    PIHOLE_DNS_9=84.200.69.80
    PIHOLE_DNS_10=84.200.70.40
    PIHOLE_DNS_11=9.9.9.9
    PIHOLE_DNS_12=149.112.112.112
    PIHOLE_DNS_13=9.9.9.10
    PIHOLE_DNS_14=149.112.112.10
    PIHOLE_DNS_15=9.9.9.11
    PIHOLE_DNS_16=149.112.112.11
    PIHOLE_DNS_17=1.1.1.1
    PIHOLE_DNS_18=1.0.0.1
    ...
    PIHOLE_DNS_1=8.8.8.8
    PIHOLE_DNS_2=8.8.4.4
1 Like