How to add multiple (bulk) blocklists via CLI in 5.0?

[techstuff]

Previously (before 5.0) you could bulk add blocklists by:
curl https://example.com | sudo tee adlists.list

What's the proper way to do it via CLI now ? I found a few threads but since it's a database thing I don't want to screw it up.

Scripting Adlist Management in v5 - #3 by papasan
Blocklist Management in PiHole v5 - #7 by jpgpi250

Someone in this thread says the following works:

sudo sqlite3 /etc/pihole/gravity.db "DELETE FROM adlist"
sudo wget -qO - https://v.firebog.net/hosts/lists.php?type=tick |xargs -I {} sudo sqlite3 /etc/pihole/gravity.db "INSERT INTO adlist (Address) VALUES ('{}');"
pihole -g

Does that look clean/safe ? I see alot of people asking how to do this so maybe you guys could add it to the GUIDE/FAQ to help others

Thanks

[papasan]

that's my code, hacked together for the same purpose. although the above works I think the best way would be to grab the new list, compare it to what's in the database, and add/remove as needed. above would fail badly if firebog.net is not available for any reason.

p.s. the first 'sudo' on the second line isn't needed.

[jpgpi250]

DELETE FROM adlist will remove all entries.
the insert command may or may not work (I didn't test it), however you'll end up with only the entries from the firebog list.

If you want to keep existing entries, and add possible new entries form the firebog list:

• don't execute the delete command
• change the INSERT command into INSERT OR IGNORE, this will prevent errors, when trying to add lists that are already in the database.

Again, I've never tested this command, it may or may not work

The example uses the ticked firebog list, I've been using the nocross firebog list. That's a choice you need to make, read here.
I have been using this script successfully. Don't forget to make the required changes (ticked or nocross). Of course, you'll need to run pihole -g after running the script.

[mmotti]

You could use a script that I've written to export as flat files, make changes and import again.

It just depends how adventurous you are.

mmotti/PyPhDB

[techstuff]

Thanks for all the ideas/input. I've settled on the @jpgpi250 script since I prefer bash. Seems to work great, I only changed the paths for "sed" (not sure if you intended /tmp to be there) and added 'pihole -g' at the end as recommended.

[papasan]

a better solution than my ham-fisted attempt, but your script is lacking in that it won't pull out any lists that drop out of the firebog repo (your script only adds, never removes).