That would be a third party script.
It has been identified as the source for a lot of issues with Pi-hole. Like locking or corrupting the database. Sometimes it even stops FTL and never restarts it leading to about a dozen reports all over the place that Pi-hole "crashed". Investigations then reveal that the logs say that FTL was shut down externally and just never got restarted. Probably gravity sync is buggy, crashes midway and leaves Pi-hole in an undefined state.
I have seen many reports that the issues didn't happen again once the cron job for gravity sync was disabled.
Got it, I also thought about this. But as I said, this was the first thing which I removed and turned off completely.
No sync is going on right now (I can also confirm this because the script is not running, removed from crontab), but I still has this issue. And since then I have tried removing the database, repairing pihole, etc...
Ok, so this one, I have stopped pihole-FTL service, fixed the IP address in the setupVars.conf (I have read somewhere that this one can be also removed, but now I left it there) removed the pihole-FTL.db and ran pihole -up.
pihole -up reported:
[i] No source list found, or it is empty
[i] Building tree...
[✗] Unable to build gravity tree in /etc/pihole/gravity.db_temp
Error: no such table: main.gravity
Error: database disk image is malformed
Error: database disk image is malformed
[i] Number of gravity domains: ( unique domains)
Error: near ")": syntax error
Error: database disk image is malformed
[i] Number of exact blacklisted domains:
Error: database disk image is malformed
[i] Number of regex blacklist filters:
Error: database disk image is malformed
[i] Number of exact whitelisted domains:
Error: database disk image is malformed
Current Pi-hole version is v5.3.1.
Current AdminLTE version is v5.5.
Current FTL version is development vDev-74a6b07.
See:
First, go back to master on FTL.
pihole checkout ftl master
Next, shut down pihole-ftl and delete (or move) the existing malformed database.
Then restart pihole-ftl.
Once the database is damaged then it's damaged. Stopping the sync wouldn't repair the existing damage and thus the database needs to be deleted completely and started over again.
Yes, I know this.
But it seemeed at some time that the database is correct, no errors like this were reported. pihole -g ran fine and I was able to use pihole normally for a few hours.
Anyway, I will move my piholes back to the master branch, delete the database on both of them, gravity-sync is turned off completely everywhere and I will use it for a few days with the included blocklist only.
I have removed all databases, ran pihole -g -r, etc, but still getting the same after a few hours.
No third party scripts are running.
https://tricorder.pi-hole.net/oaxx8hgajp
Something is still running:
[2021-04-23 17:50:02.825 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:03.840 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:03.840 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:05.438 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:05.439 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:07.439 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:07.439 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:09.446 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:09.447 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:11.444 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:11.444 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:12.444 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:12.445 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:12.448 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:12.448 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:13.445 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:13.445 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:15.449 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:15.449 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:17.450 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:17.451 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:19.451 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:19.452 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:21.438 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:21.439 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:22.455 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:22.456 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:22.468 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:22.469 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:22.471 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:22.472 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:23.471 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:23.472 13129/T13130] IPv4 telnet error: Success (0)
[2021-04-23 17:50:25.443 13129/T13130] Client denied (at max capacity of 255): 255
[2021-04-23 17:50:25.444 13129/T13130] IPv4 telnet error: Success (0)
There also appear to be numerous repeated queries for:
http://127.0.0.1:80/admin/scripts/pi-hole/php/queryads.php?domain=o.ss2.us&bp
2021-04-19 09:56:09: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_filter() expects parameter 1 to be array, string given in /var/www/html/pihole/index.php on line 173
2021-04-19 09:56:09: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_values() expects parameter 1 to be array, null given in /var/www/html/pihole/index.php on line 173
2021-04-19 09:56:10: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: file(http://127.0.0.1:80/admin/scripts/pi-hole/php/queryads.php?domain=o.ss2.us&bp): failed to open stream: HTTP request failed! in /var/www/html/pihole/index.php on line 172
2021-04-19 09:56:10: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_filter() expects parameter 1 to be array, string given in /var/www/html/pihole/index.php on line 173
2021-04-19 09:56:10: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_values() expects parameter 1 to be array, null given in /var/www/html/pihole/index.php on line 173
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: file(http://127.0.0.1:80/admin/scripts/pi-hole/php/queryads.php?domain=o.ss2.us&bp): failed to open stream: HTTP request failed! in /var/www/html/pihole/index.php on line 172
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_filter() expects parameter 1 to be array, string given in /var/www/html/pihole/index.php on line 173
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_values() expects parameter 1 to be array, null given in /var/www/html/pihole/index.php on line 173
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: file(http://127.0.0.1:80/admin/scripts/pi-hole/php/queryads.php?domain=o.ss2.us&bp): failed to open stream: HTTP request failed! in /var/www/html/pihole/index.php on line 172
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_filter() expects parameter 1 to be array, string given in /var/www/html/pihole/index.php on line 173
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_values() expects parameter 1 to be array, null given in /var/www/html/pihole/index.php on line 173
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: file(http://127.0.0.1:80/admin/scripts/pi-hole/php/queryads.php?domain=o.ss2.us&bp): failed to open stream: HTTP request failed! in /var/www/html/pihole/index.php on line 172
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_filter() expects parameter 1 to be array, string given in /var/www/html/pihole/index.php on line 173
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_values() expects parameter 1 to be array, null given in /var/www/html/pihole/index.php on line 173
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: file(http://127.0.0.1:80/admin/scripts/pi-hole/php/queryads.php?domain=o.ss2.us&bp): failed to open stream: HTTP request failed! in /var/www/html/pihole/index.php on line 172
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_filter() expects parameter 1 to be array, string given in /var/www/html/pihole/index.php on line 173
2021-04-19 09:56:13: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: array_values() expects parameter 1 to be array, null given in /var/www/html/pihole/index.php on line 173
Yes I can also see this. But honestly, there are no active third party scripts right now.
Right now, it really seems for me that the SD card is near its death (that might also explain the database corruption!? - but what about the telnet errors?)
You'll have to find out what process is trying to telnet in to the API on port 4711 every two seconds. It's not something we do. Tools that you can try are ps or sudo ps and ss or sudo ss.
I didn't find anything...
Anyway I have restored a backup, restarted the Raspberry once and now it seems to run fine... (on the old version).
Right now it seems for me that nothing special can be found in the logs.
I will try to update it again, soon
Same problem at my raspberry pi 4 after last pi hole update. Please help to solve. Try restart (pihole restartdns) this solve problem for 15minutes and web interface crash after.
debug log from my device:
You're affected by the same issue
[2021-04-26 10:30:00.353 10207/T10211] Encountered step error in get_max_query_ID(): database disk image is malformed
You can either remove the database or we can attempt a recovery.
The recovery procedure would be
pihole checkout ftl tweak/sqlite3_recover
sudo service pihole-FTL stop
pihole-FTL /etc/pihole/pihole-FTL.db ".recover" | pihole-FTL ~/pihole-FTL_recovered.db
sudo mv ~/pihole-FTL_recovered.db /etc/pihole/pihole-FTL.db
sudo service pihole-FTL start
if you are willing to test this. If it doesn't work, the database needs to be removed and everything will go back to working again (but you'd loose historic data).
2 Likes
Well I have restored the backup, ran fine and I updated again to the latest version.
Now everything seems to work normally, no errors in the log, even gravity-sync seems to work fine.
1 Like
After recovery procedure for now working normal. Thanks.
1 Like
I am also getting the error.
Version Details Pi-hole v5.3.1, Web Interface 5.5, FTL v5.8.1,
That should I do, as I am running DietPi OS?
thanks. your procedure sorted out problem for me.
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.