FTL is crashing everytime data is saved to the FTL database.
My DBINTERVAL was set to 30... now set to 5 to test.
I tried to reboot my RPI.
I also tried to delete FTL DB file.
I'm using raspiBackup to do my daily backup on an external USB key.
this crash started happening right after the backup. It worked fine all day before this backup.
to make sure the FTL DB is consistent, I stop the FTL service before the backup.
adding some info about my system :
uname -a
Linux raspberrypi 4.19.97-v7l+ #1294 SMP Thu Jan 30 13:21:14 GMT 2020 armv7l GNU/Linux
pihole -v
Pi-hole version is v4.3.5-459-g0fad979 (Latest: v4.4)
AdminLTE version is v4.3.2-439-g6fdeee5 (Latest: v4.3.3)
FTL version is vDev-b6364d0 (Latest: v4.3.1)
/proc/cpuinfo
Hardware : BCM2835
Revision : c03111
Model : Raspberry Pi 4 Model B Rev 1.1
I killed my entire Sunday trying to figure this one out. Ended up reverting back to stable, even though i've been running 5.0 since the day it was announced. Fixed a bunch of other things in the process, so i got that going for me...
2020-03-30 03:01:00.728 8956] dbquery: "SELECT id FROM network WHERE hwaddr = '68:c6:3a:af:2c:d4';"
[2020-03-30 03:01:00.728 8956] Querying gravity database for client 10.0.1.119 (counting)
[2020-03-30 03:01:00.729 8956] Querying gravity database for client 10.0.1.119 (counting)
[2020-03-30 03:01:00.730 8956] dbquery: "SELECT id FROM network WHERE hwaddr = 'ip-10.0.1.119' AND firstSeen > (cast(strftime('%s', 'now') as int)-3600);"
[2020-03-30 03:01:00.755 8956] Device with IP 10.0.1.119 not known and no recent mock-device found ---> creating new record
[2020-03-30 03:01:00.755 8956] dbquery: "INSERT INTO network (hwaddr,interface,firstSeen,lastQuery,numQueries,name,macVendor) VALUES ('68:c6:3a:af:2c:d4','enp2s0',4294967297, 0, 0, '', 'Espressif Inc.');"
[2020-03-30 03:01:00.756 8956] dbquery: "INSERT OR REPLACE INTO network_addresses (network_id,ip,lastSeen) VALUES(21,'10.0.1.119',(cast(strftime('%s', 'now') as int)));"
[2020-03-30 03:01:00.756 8956] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-03-30 03:01:00.756 8956] ----------------------------> FTL crashed! <----------------------------
[2020-03-30 03:01:00.756 8956] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-03-30 03:01:00.756 8956] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-03-30 03:01:00.757 8956] and include in your report already the following details:
[2020-03-30 03:01:00.757 8956] FTL has been running for 18 seconds
[2020-03-30 03:01:00.757 8956] FTL branch: release/v5.0
[2020-03-30 03:01:00.757 8956] FTL version: vDev-b6364d0
[2020-03-30 03:01:00.757 8956] FTL commit: b6364d0
[2020-03-30 03:01:00.757 8956] FTL date: 2020-03-29 23:23:02 +0200
[2020-03-30 03:01:00.757 8956] FTL user: started as pihole, ended as pihole
[2020-03-30 03:01:00.758 8956] Compiled for x86_64 (compiled on CI) using gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
[2020-03-30 03:01:00.758 8956] Received signal: Segmentation fault
[2020-03-30 03:01:00.758 8956] at address: 0x7f4d00000001
[2020-03-30 03:01:00.758 8956] with code: SEGV_MAPERR (Address not mapped to object)
[2020-03-30 03:01:00.759 8956] Backtrace:
[2020-03-30 03:01:00.759 8956] B[0000] 0x564f734b3189, /usr/bin/pihole-FTL(+0x31189) [0x564f734b3189]
[2020-03-30 03:01:00.760 8956] B[0001] 0x7f4d5de97890, /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890) [0x7f4d5de97890]
[2020-03-30 03:01:00.760 8956] B[0002] 0x7f4d5dbc8452, /lib/x86_64-linux-gnu/libc.so.6(__vasprintf_chk+0x102) [0x7f4d5dbc8452]
[2020-03-30 03:01:00.760 8956] B[0003] 0x7f4d5dbc832f, /lib/x86_64-linux-gnu/libc.so.6(__asprintf_chk+0x8f) [0x7f4d5dbc832f]
[2020-03-30 03:01:00.760 8956] B[0004] 0x564f734a8c78, /usr/bin/pihole-FTL(parse_neighbor_cache+0x248) [0x564f734a8c78]
[2020-03-30 03:01:00.760 8956] B[0005] 0x564f734ac87d, /usr/bin/pihole-FTL(DB_thread+0x10d) [0x564f734ac87d]
[2020-03-30 03:01:00.760 8956] B[0006] 0x7f4d5de8c6db, /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f4d5de8c6db]
[2020-03-30 03:01:00.760 8956] B[0007] 0x7f4d5dbb588f, /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f4d5dbb588f]
[2020-03-30 03:01:00.760 8956] ------ Listing content of directory /dev/shm ------
[2020-03-30 03:01:00.760 8956] File Mode User:Group Filesize Filename
[2020-03-30 03:01:00.761 8956] rwxrwxrwx root:root 260 .
[2020-03-30 03:01:00.761 8956] rwxr-xr-x root:root 4K ..
[2020-03-30 03:01:00.761 8956] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-03-30 03:01:00.762 8956] rw------- pihole:pihole 4K FTL-dns-cache
[2020-03-30 03:01:00.762 8956] rw------- pihole:pihole 12K FTL-overTime
[2020-03-30 03:01:00.762 8956] rw------- pihole:pihole 524K FTL-queries
[2020-03-30 03:01:00.763 8956] rw------- pihole:pihole 4K FTL-upstreams
[2020-03-30 03:01:00.763 8956] rw------- pihole:pihole 340K FTL-clients
[2020-03-30 03:01:00.763 8956] rw------- pihole:pihole 98K FTL-domains
[2020-03-30 03:01:00.764 8956] rw------- pihole:pihole 12K FTL-strings
[2020-03-30 03:01:00.764 8956] rw------- pihole:pihole 12 FTL-settings
[2020-03-30 03:01:00.764 8956] rw------- pihole:pihole 124 FTL-counters
[2020-03-30 03:01:00.765 8956] rw------- pihole:pihole 48 FTL-lock
[2020-03-30 03:01:00.765 8956] ---------------------------------------------------
[2020-03-30 03:01:00.765 8956] Thank you for helping us to improve our FTL engine!
[2020-03-30 03:01:00.765 8956] FTL terminated!
Same here, not using DHCP on Pi-Hole.
Using Filewalla in simple mode with DHCP on my provider-supplied router still enabled.
While I now see graphs on Pi-Hole, no blocking seems to be indicated there.
Furthermore, when doing some tests, results seem to vary (using standard
For instance, doubleclick.net now goes straight to the relevant page on google when disabling ublock origin on Firefox while the command
Yesterday (on vDev-b6364d0) it was crashing for me every 60-120 seconds. I updated to the new release today (vDev-81c4eac), and now it is crashing every 15 minutes on the dot. Which lines up with my DBINTERVAL.
Not using DHCP, crashes if I keep the dashboard open. Something similar happened several weeks ago, close the dashboard in the browser and let it run in the background and it's stable...at least for me it is.
I thought it might be related to the dhcp server because it seemed FTL crashed right after I clicked save when enabling dhcp. Could there be something else in the dhcp settings in the web interface that is causing the problem? Maybe a static address that needs to be saved?
They still need a bit of updating, in case the debugger halts on any other signal than SIGSEGV (for instance on a real-time signal) then this is likely not a bug but intended for updating the lists cache or similar. You can just issue continue in this case and FTL will continue to run.
When you hit the bug, please ask the debugger for a backtrace and put your findings here. I will go through them with you later today, asking more specifically for more details depending on whatever anyone of you finds.
Thank you all for your patience and for helping us out with FTL. The beta testing proves very fruitful, once again, as FTL is crashing for noone of the "core" developers.
Thread 5 "database" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb4e92460 (LWP 27555)]
__strchrnul (s=0x1 <error: Cannot access memory at address 0x1>, c_in=37) at strchrnul.c:50
50 strchrnul.c: Datei oder Verzeichnis nicht gefunden.
(gdb) backtrace
#0 __strchrnul (s=0x1 <error: Cannot access memory at address 0x1>, c_in=37) at strchrnul.c:50
#1 0xb6d7f174 in __find_specmb (format=0x1 <error: Cannot access memory at address 0x1>) at printf-parse.h:108
#2 _IO_vfprintf_internal (s=s@entry=0xb4e91a90, format=format@entry=0x1 <error: Cannot access memory at address 0x1>, ap=..., ap@entry=...) at vfprintf.c:1315
#3 0xb6e27024 in __GI___vasprintf_chk (result_ptr=result_ptr@entry=0xb4e91c60, flags=flags@entry=1, format=0x1 <error: Cannot access memory at address 0x1>, format@entry=0x0, args=..., args@entry=...) at vasprintf_chk.c:66
#4 0xb6e26f30 in __asprintf_chk (result_ptr=result_ptr@entry=0xb4e91c60, flags=flags@entry=1, format=0x1 <error: Cannot access memory at address 0x1>) at asprintf_chk.c:32
#5 0x00479b2a in asprintf (__fmt=0x5645d4 "SELECT id FROM network WHERE hwaddr = '%s';", __ptr=0xb4e91c60) at /usr/arm-linux-gnueabihf/include/bits/stdio2.h:178
#6 parse_neighbor_cache () at src/database/network-table.c:382
#7 0x0047cae0 in DB_thread (val=<optimized out>) at src/database/database-thread.c:68
#8 0xb6e92494 in start_thread (arg=0xb4e92460) at pthread_create.c:486
#9 0xb6e15578 in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)
Thread 5 "database" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7522f460 (LWP 2335)]
0x76dfc428 in strchrnul () from /lib/arm-linux-gnueabihf/libc.so.6
(gdb) backtrace
#0 0x76dfc428 in strchrnul () from /lib/arm-linux-gnueabihf/libc.so.6
#1 0x76dc2174 in vfprintf () from /lib/arm-linux-gnueabihf/libc.so.6
#2 0x76e6a024 in __vasprintf_chk () from /lib/arm-linux-gnueabihf/libc.so.6
#3 0x76e69f30 in __asprintf_chk () from /lib/arm-linux-gnueabihf/libc.so.6
#4 0x004d5b2a in asprintf (__fmt=0x5c05d4 "SELECT id FROM network WHERE hwaddr = '%s';", __ptr=0x7522ec60)
at /usr/arm-linux-gnueabihf/include/bits/stdio2.h:178
#5 parse_neighbor_cache () at src/database/network-table.c:382
#6 0x004d8ae0 in DB_thread (val=<optimized out>) at src/database/database-thread.c:68
#7 0x76ed5494 in start_thread () from /lib/arm-linux-gnueabihf/libpthread.so.0
#8 0x76e58578 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(hope pasting it a second time helps in any way, didn't want to flood the thread with unneccessary information, but as you asked "all" to put FTL into the debugger…)
Running Pihole on DietPi v6.28.0/RPi 2 Model B, connection via Ethernet, Wifi disabled.
Adding MAXDBDAYS=0 to /etc/pihole/setupVars.conf seems to solve the issue as a workaround.