Create FTL tree on PC, copy to Pi-hole device

[il-mix]

I just built Pi-hole with my old Raspberry Pi B from 2012. It works well enough as a DNS+DHCP server. The resources seems enough for standard operations, but it fails/crashes when building the list tree.

I’d like to build the tree on a PC with the same lists used on the pi-hole, and copy the .db file on the Raspberry.

Can I do this?

What do I need to do to have the Pi-hole use the updated db? Is it enough to copy it or do I need to launch any update/refresh command?

On PC side, what do I need to install to build the tree? The whole Pi-hole stack, or can I just install FTL? Or even just launch a couple of scripts with the lists as input, without installing anything?

Thanks for your help!

[deHakkelaar]

@il-mix

fails/crashes when building the list tree.

I have two Pi 1B's that dont crash upon a gravity update (one with and one without the +):

$ cat /proc/device-tree/model; echo
Raspberry Pi Model B Rev 1

$ cat /proc/device-tree/model; echo
Raspberry Pi Model B Rev 2

I'd say focus on getting it to work on the Pi instead.
I have some tips in below link like for example expanding memory a bit to keep the OOM reaper out of the door:

Prepping a Raspberry Pi 1B for Pi-hole with Raspbian 12 Bookworm

I noticed some users asking for more detailed instructions for installing Pi-hole on a Raspberry Pi or another Linux setup. As I was in the process of reinstalling both my trusty Pi 1B's with Raspbian Bookworm and the latest Pi-hole v6 release, and usually I document the steps, I thought why not share here. Both Pi's have served Pi-hole reliable for over eight years now, still running on the original SD cards, and I think they have many years to come. As its a headless setup, I'll be doing so…

And depending Pi model, you can check with below for under-voltage messages that can crash your Pi and worst case can corrupt the filesystem:

sudo journalctl --full --no-hostname --pager-end --grep voltage

FYI:

https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#power-supply-warnings

[il-mix]

I must say that I have done some tweaks to have it (almost) in read-only mode, following this

GitHub - jacklul/pihole-readonly-rootfs: Installing Pi-hole on read-only rootfs system to...

So, some RAM is already used for the OS. Although it doesn’t seem too much. Here is the output of ‘free -m’
total used free shared buff/cache available
Mem: 176 39 83 5 52 83
Swap: 0 0 0

Extra info, the Rasbberry is a model B Rev 1 (thanks for the command).

I read the links you gave me and try to fix everything. Will probably be better to start anew with a clean system.

[deHakkelaar]

@il-mix

I must say that I have done some tweaks to have it (almost) in read-only mode

You could have mentioned that valuable info in the OP!

Below bit on Github is wrong/old:

sudo nano /boot/config.txt

# Reduce memory reserved for GPU
gpu_mem=16

Or are you still running Buster/Bullseye?
On my Pi with Pi-OS 12 Bookworm, that file explicitly says:

$ head /boot/config.txt
DO NOT EDIT THIS FILE

The file you are looking for has moved to /boot/firmware/config.txt

You can find it on below mount/partition:

$ findmnt /boot/firmware
TARGET         SOURCE         FSTYPE OPTIONS
/boot/firmware /dev/mmcblk0p1 vfat   rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro

@il-mix

total used free shared buff/cache available
Mem: 176 39 83 5 52 83
Swap: 0 0 0

Below is on my 256MB Pi 1B (without the plus) when you have it right:

$ vcgencmd get_mem gpu; vcgencmd get_mem arm
gpu=16M
arm=240M

$ free -h
               total        used        free      shared  buff/cache   available
Mem:           221Mi       125Mi        39Mi       4.6Mi       111Mi        96Mi
Swap:          511Mi        31Mi       480Mi

This Pi also runs Unbound for upstream!

But since you experience crashes, and the Pi 1B without the plus doesnt have the voltage check, the power supply is still a suspect!
Or the USB cable powering the Pi is of poor quality.
Or maybe you have power hungry USB devices connected?
As explained in that Raspi link.

[il-mix]

I’m running Bullseye. I opted for Raspbian Legacy version, since the hw is kinda "legacy”, too.

I skipped the set of the gpu memory in the config file because I thought “hey, it’s a headless system, I’m not unsing the GPU anyway”. Turns out the gpu does some extra things, and still has memory allocated by default.
Setting the GPU parameter I’ll get extra memory, indeed:

           `total        used        free      shared  buff/cache   available`

Mem: 223 37 80 5 105 132
Swap: 0 0 0

Swap is still disabled, as a way to preserve the SD.

Regarding the power supply, I have wired a barrel connector to the box, so the connection is quite sturdy. The power supply is a 5V, 1.2A. Enough power imho.
In the posted read-only guide there is an extra setting for the watchdog.

RuntimeWatchdogSec=15
RebootWatchdogSec=5min
ShutdownWatchdogSec=5min

I suppose this is what reboots the device when it hangs building the tree. Have to check some system logs.

[il-mix]

UPDATE

With the gpu memory “downgrade” tip, so freeing some more memory for the CPU, I already got better results. I was able to generate the tree adding a couple of lists that caused the process to fail before.

[deHakkelaar]

@il-mix

I’m running Bullseye.

If setup a new instance, why not go for the lates?
It wont take long before Bullseye (11) wont be supported anymore with Trixie being released already.

$ dig +short versions.pi-hole.net txt
"Raspbian=11,12 Ubuntu=20,22,23,24 Debian=11,12 Fedora=40,41 CentOS=9,10"

About power supplies, they have been the main cause of troubles on most of my Raspi/O-droid/Tinkerboard projects.

[il-mix]

May I ask you how many blocked domains you have on your Raspberry Pi 1?
As said, after twaking the GPU memory I was able to correctly build the database. But it looks like it caps at ~1.000.000 domains. When I add that many domains (in multiple lists, or a single big list), the database build process fails.

I still don’t have swap enabled, so I have a little less available RAM (or pseudo-RAM) available than you. But I’m curious about how much I’ll be able to squeeze out from the poor RPi if I tweak a couple more things (e.g. enable swap).

[deHakkelaar]

@il-mix

May I ask you how many blocked domains you have on your Raspberry Pi 1?

Only the default + a very small list currently (230k).
I remember at some stage I used to have little over 500k on the old Pi-hole version 5.
But not for long.
I cant imagine the v6 release requiring much more memory to build the gravity db.
As you can see from my free -h, I still have 96MB available memory to build the db in RAM which should be more than enough for normal use.

Its just pointless to have that many in the db.
You're only hitting a very small portion from that humongous list of domains.
And the more lists, the more false positives that need your attention.

@il-mix

But it looks like it caps at ~1.000.000 domains.

Check the OOM killer:

sudo journalctl --full --no-hostname --pager-end --grep oom

Or if it says --grep isnt compiled:

sudo journalctl --full --no-hostname | grep -i oom

Plus check the other logs at the time of those crashes to diagnose properly (journalctl, Pi-hole logs etc).

But you're skipping my question?

@deHakkelaar

If setup a new instance, why not go for the lates?

[il-mix]

Thanks for the info. As said I’m working with bigger lists. Maybe I need to cut some (ex. use smaller versions of Hagezi’s lists).

EDIT: enabling swap (250MB) I was able to generate a 1.101.760 domains list

…and yes, I’m skipping your question
I have a super-tight fit case (I had to remove USB and audio connector and rewire the SD socket). At the moment I don’t want to unpack everything to install the upgraded version.
I’ll probably do it in the near future, after some more testing.

[deHakkelaar]

Post a link for one of those big lists that crashes and I'll try to load it on my Pi?