Multiple regex lists

General
,
1

I found a regex list on reddit, it's in the first comment.

This list seems to change regularly, I also want to use my own list (converted wildcard list - pihole 3.x), so in order to combine these, without too much work, I wrote the following script.

The script uses a ram drive (reduce SD card writing) to process the lists, if You want to use a ram drive, you need to create it, before running the script. To create the ram drive (run this script only once):

#!/bin/bash

# Make sure only root can run our script
if [ "$(id -u)" != "0" ]; then
  echo "This script must be run as root" 1>&2
  exit 1
fi

# ramdrive
mkdir /home/pi/tmp
sudo sed -i '$ a tmpfs /home/pi/tmp tmpfs nodev,nosuid,size=64M 0 0' /etc/fstab

If you don't want to use a ramdrive, simply change the paths...

In order to use my (converted wildcard) regex list in the script, I've put it in /var/www/html (the pihole webserver), so it can be downloaded by the script.

The script downloads both lists and combines the lists. It the copies the result to /etc/pihole and restarts pihole-FTL to process them

#!/bin/bash

# Make sure only root can run our script
if [ "$(id -u)" != "0" ]; then
  echo "This script must be run as root" 1>&2
  exit 1
fi

mount /home/pi/tmp
curl https://raw.githubusercontent.com/mmotti/pihole-regex/master/regex.list -o /home/pi/tmp/regex1.txt
chmod 666 /home/pi/tmp/regex1.txt
echo >> /home/pi/tmp/regex1.txt
# this is my pihole's IP address, replace it with your own!!!
wget http://192.168.2.57/regex.list -O /home/pi/tmp/regex2.txt
cat /home/pi/tmp/regex1.txt /home/pi/tmp/regex2.txt > /home/pi/tmp/regex.txt
sort /home/pi/tmp/regex.txt | uniq -u > /home/pi/tmp/regex.list
sed -i '/^$/d' /home/pi/tmp/regex.list
sed -i '/^#/d' /home/pi/tmp/regex.list
mv /home/pi/tmp/regex.list /etc/pihole/regex.list
umount /home/pi/tmp
service pihole-FTL stop
service pihole-FTL start

notes:

  • to add a list, simply download it to the next available temp file (regex3.txt) and add it to the 'cat' line.
  • not all downloaded list end with a new line, so add an empty line to the list, using echo
  • automate the script by creating an new file in /etc/cron.d

I'm sure this script can be improved, works for me...
@DL6ER: Feature Request: Allow multiple rexex lists, download and process them with 'pihole-g'

2

This is not so simple as

is not necessarily sufficient to remove duplicates, e.g.

((^)|(\.))twitter.com
(^)|(\.)twitter.com

Both regex are perfectly valid any subject to the personal preference of the list's creator.

3

The script is working as far as creating and moving the regex file to /etc/pihole/regex.list
However, pihole-FTL isn't restarting when I run it via cron.
Pihole-FTL rstarts when I run the script via the cmd though.

6

If you want to run the script with cron:

  • create a cron job (change time and scriptname)
50 10    * * *   root    PATH="$PATH:/home/pi/" /home/pi/test.sh
  • replace the service commands (last two lines) with the following lines:
systemctl stop pihole-FTL.service
systemctl start pihole-FTL.service

tested, should work ...

7

That's not how feature request tickets work :wink:

9

Great. Thanks. Not adding the 'root' worked for me.

50 10  * * *   PATH="$PATH:/home/pi/" /home/pi/test.sh