SQL error prepare: database is locked :(

Expected Behaviour:

Running an standard installation that listens on both wired and wireless interfaces. PiHole works but hangs after a few days.

Actual Behaviour:

FTL log keeps showing errors about the database being locked, for example "SQLite3 message: database is locked in "SELECT hwaddr FROM network WHERE id = (SELECT network_id FROM network_addresses WHERE ip = ? GROUP BY ip HAVING max(lastSeen));" (5)"

Tried to reboot but it doesn't help.

Debug Token:

https://tricorder.pi-hole.net/c69vVXdj/

Thank you! :pray:

Probably unrelated to your issue:

Why would you do so when your debug log shows that both interfaces are on the same subnet?

*** [ DIAGNOSING ]: Network interfaces and addresses
   2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
       inet 192.168.0.200/24 brd 192.168.0.255 scope global noprefixroute eth0
          valid_lft forever preferred_lft forever
   3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
       inet 192.168.0.199/24 brd 192.168.0.255 scope global noprefixroute wlan0
          valid_lft forever preferred_lft forever

As your RPi is accessible via a wired eth0 interface, that should be your preferred connection.

You may as well turn off your RPi's wlan0 interface completely and spare your network from some routing decisions and save some power as well.

You'd have to run pihole -r with Reconfigure if you'd have currently tied your Pi-hole to wlan0 instead of eth0.

Also, your debug log shows you are using as much as fourteen different upstream DNS servers:

*** [ DIAGNOSING ]: Setup variables
    PIHOLE_DNS_1=8.8.8.8
    PIHOLE_DNS_2=8.8.4.4
    PIHOLE_DNS_3=2001:4860:4860:0:0:0:0:8888
    PIHOLE_DNS_4=2001:4860:4860:0:0:0:0:8844
    PIHOLE_DNS_5=208.67.222.222
    PIHOLE_DNS_6=208.67.220.220
    PIHOLE_DNS_7=2620:119:35::35
    PIHOLE_DNS_8=2620:119:53::53
    PIHOLE_DNS_9=1.1.1.1
    PIHOLE_DNS_10=1.0.0.1
    PIHOLE_DNS_11=2606:4700:4700::1111
    PIHOLE_DNS_12=2606:4700:4700::1001
    PIHOLE_DNS_13=168.95.192.1
    PIHOLE_DNS_14=168.95.1.1

You'd usually pick upstreams based on certain properties, like resolution speed, or if they'd not log your DNS activities, or if they would offer DNSSEC support, or if they'd apply additional filtering, etc...

Mixing them all together like you did may give you mixed resolution results depending on the actual upstream that Pi-hole would be using for a given DNS query.

You should revisit your selection of upstreams and restrict them to those two or four that best match your personal preferences.

Back to your issue:
Your database would get locked on write access, as a measure to prevent multiple write requests (potentially from different processes) from corrupting your data.
This can happen e.g. if you try to update the database via some UI interaction while pihole-FTL would be writing to it at the same time. So this is expected to occur at times.
Usually, repeating your UI request after some time should allow it to succeed.

In your case, however, your log shows a peculiar irregularity:

*** [ DIAGNOSING ]: Pi-hole log
-rw-r----- 1 pihole pihole 887K Jul  5 13:16 /var/log/pihole/pihole.log

   -----tail of pihole.log------
   Jul  5 13:16:28 dnsmasq[6657]: forwarded gql-realtime.reddit.com to 8.8.8.8
   Jul  5 13:16:28 dnsmasq[6657]: reply gql-realtime.reddit.com is <CNAME>
   Jul  5 13:16:28 dnsmasq[6656]: forwarded gql-realtime.reddit.com to 8.8.8.8
   Jul  5 13:16:28 dnsmasq[6657]: reply prod-3-realtime-lb-840806869.us-east-1.elb.amazonaws.com is 54.209.177.175
   Jul  5 13:16:28 dnsmasq[6656]: reply gql-realtime.reddit.com is <CNAME>
   Jul  5 13:16:28 dnsmasq[6657]: reply prod-3-realtime-lb-840806869.us-east-1.elb.amazonaws.com is 18.207.82.243
   Jul  5 13:16:28 dnsmasq[6656]: reply prod-3-realtime-lb-840806869.us-east-1.elb.amazonaws.com is NODATA
   Jul  5 13:16:28 dnsmasq[6657]: reply prod-3-realtime-lb-840806869.us-east-1.elb.amazonaws.com is 34.227.54.142
   Jul  5 13:16:28 dnsmasq[6657]: reply prod-3-realtime-lb-840806869.us-east-1.elb.amazonaws.com is 23.20.50.148
   Jul  5 13:16:28 dnsmasq[6657]: reply prod-3-realtime-lb-840806869.us-east-1.elb.amazonaws.com is 54.146.39.97
   Jul  5 13:16:28 dnsmasq[6657]: reply prod-3-realtime-lb-840806869.us-east-1.elb.amazonaws.com is 34.197.194.240
   Jul  5 13:16:28 dnsmasq[6657]: reply prod-3-realtime-lb-840806869.us-east-1.elb.amazonaws.com is 3.228.166.145
   Jul  5 13:16:28 dnsmasq[6657]: reply prod-3-realtime-lb-840806869.us-east-1.elb.amazonaws.com is 3.222.24.15
   Jul  5 13:16:31 dnsmasq[541]: query[A] BRWDCE99452F787.iis.sinica.edu.tw from 192.168.0.201
   Jul  5 13:16:31 dnsmasq[541]: cached BRWDCE99452F787.iis.sinica.edu.tw is NXDOMAIN
   Jul  5 13:16:36 dnsmasq[7743]: query[HTTPS] signaler-pa.clients6.google.com from 192.168.0.201
   Jul  5 13:16:36 dnsmasq[7744]: query[A] signaler-pa.clients6.google.com from 192.168.0.201
   Jul  5 13:16:36 dnsmasq[7744]: cached signaler-pa.clients6.google.com is 142.251.43.10
   Jul  5 13:16:36 dnsmasq[7743]: forwarded signaler-pa.clients6.google.com to 8.8.8.8
   Jul  5 13:16:36 dnsmasq[7743]: reply signaler-pa.clients6.google.com is NODATA

Over a timespan of 8 seconds, there are five different PIDs involved (6656, 6657, 541, 7743 and 7744).
By itself, this may not be unusual, as Pi-hole could have forked processes to handle the queries in parallel.

But together with your observation, it may also suggest that there may be multiple unsolicited instances of pihole-FTL running at the same time.

Run from your Pi-hole machine, what's the output of:

sudo ps -eF | grep "PID\|pihole\|dnsmasq"
And certainly not related to your issue, but you should check your router's IPv6 DNS configuration (click for more)

Your debug log shows that your RPi hosting Pi-hole has picked up two IPv6 DNS server addresses:

*** [ DIAGNOSING ]: contents of /etc

-rw-r--r-- 1 root root 82 Jun 17 08:45 /etc/resolv.conf
   nameserver 2001:b000:168::2
   nameserver 2001:b000:168::1

Those are IPv6s associated with dns.hinet.net, operated by Chunghwa Telecom (probably your ISP?).

Chances are your RPi learned them from your router.
If that would be the case, all your network's IPv6 capable clients would be able to by-pass your Pi-hole using those IPv6 DNS servers.
You'd have to stop your router from advertising those IPv6 addresses as DNS servers.

Thanks for the detailed answer!
I am running both the wired and the wireless interfaces because I want the slightly lower latency of the wired interface when directly connected to the router but I also want to be able to move my RPi to another location away from the router from time to time. And I want this flexibility without reconfiguring the box.

Regarding FTL, it seems to have only one instance:

sudo ps -eF | grep "PID\|pihole\|dnsmasq"

Here is the output:

UID        PID  PPID  C    SZ   RSS PSR STIME TTY          TIME CMD
pihole     541     1  0 43056 12116   2 13:05 ?        00:01:59 /usr/bin/pihole-FTL -f
rpi      22261 22249  0  1896   568   1 17:56 pts/0    00:00:00 grep --color=auto PID\|pihole\|dnsmasq

Regarding IPv6, I just disabled it on the router. And I also reduced the number of upstream DNS servers to 2. Thanks again!

Let's check pihole-FTLs children next:

sudo pstree -aps 541

rpiuser@raspberrypi:~ $ sudo ps -eF | grep "PID\|pihole\|dnsmasq"
UID        PID  PPID  C    SZ   RSS PSR STIME TTY          TIME CMD
rpiuser   6766  5729  0  1863   552   1 18:50 pts/1    00:00:00 grep --color=auto PID\|pihole\|dnsmasq
pihole   25300     1  0 42797 10652   1 18:07 ?        00:00:17 /usr/bin/pihole-FTL -f
rpiuser@raspberrypi:~ $ sudo pstree -aps 25300
systemd,1 splash
  └─pihole-FTL,25300 -f
      ├─{pihole-FTL},25319
      ├─{pihole-FTL},25320
      ├─{pihole-FTL},25321
      ├─{pihole-FTL},25322
      ├─{pihole-FTL},25323
      ├─{pihole-FTL},25324
      ├─{pihole-FTL},25325
      ├─{pihole-FTL},25326
      ├─{pihole-FTL},25327
      ├─{pihole-FTL},25328
      ├─{pihole-FTL},25329
      ├─{pihole-FTL},25330
      ├─{pihole-FTL},25331
      ├─{pihole-FTL},25332
      ├─{pihole-FTL},25333
      ├─{pihole-FTL},25334
      ├─{pihole-FTL},25335
      └─{pihole-FTL},25336
rpiuser@raspberrypi:~ $

Ah, the PID changed, so you've restarted Pi-hole in the meantime.
We won't know whether the PIDs from the log were forked children of your main 541 pihole-FTL now.
But given that we have seen only one pihole-FTL process at any time, it can be assumed that there have been no rogue processes. :wink:

Are you still observing those database locked messages?
Do you consistently observe them when interacting with Pi-hole's UI?
If so, what UI interaction triggers your observation?

Actually I didn't restart PiHole but I did change the configuration (upstream servers). Perhaps that restarted the process?

What triggered my observation is that every few days/weeks PiHole gets hung, responding to pings but not much else. This led me to checking the logs. I'll keep observing the log and report back in a few days :slight_smile:

As explained, database locks can be expected to occur.
By themselves, they would not necessarily mean that something went terribly wrong.

What makes you suspect that your issue is related to the database locked messages?
Do they match the times when you observe those freezes?

Also, you should check DNS resolution when you observe such a freeze, e.g. by running the following commands from a client:

nslookup pi.hole
nslookup flurry.com 192.168.0.200

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.