Blocking via regex now available in FTLDNS

Announcements
1

Originally published at: Blocking via regex now available in FTLDNS – Pi-hole

We have implemented GNU Extended Regular Expressions for blocking domains into FTLDNS (as used by popular tools such as egrep (or grep -E ...), awk, and emacs).

To try it, you need to be participating in the FTLDNS beta test (see here for more details). This is a new feature and we invite you to test it out but you should expect some rough edges. We would also appreciate if you could help us find any bugs or issues you run into.

Reach out to us on Discourse or Reddit with any issues you run into.

Once you're on the beta testing branch you can configure the regex of your choice in /etc/pihole/pihole-FTL.conf. In contrast to our already existing wildcard blocking implementation, you can now configure arbitrarily complex blocking filters with Pi-hole FTLDNS. The following regex:

BLOCKINGREGEX=^ab.+\.com$

will block all domains that start with "ab" (^ab), have at least one further character (.+) and end in ".com" (.com$).

Examples for what would be blocked by this rule:

  • abc.com
  • abtest.com
  • ab.test.com
  • abr-------.whatever.com
Examples for what would not be blocked by this rule:
  • testab.com (the domain doesn't start with "ab")
  • tab.test.com (the domain doesn't start with "ab")
  • ab.com (there is no character in between "ab" and ".com")
  • test.com.something (the domain doesn't end in ".com")
Hopefully this illustrates how powerful the new blocking method of FTLDNS is but also why testing its mandatory to ensure it is working correctly in all possible situations. The potential of this new blocking is huge and may even help with things like this.
5 Likes
3

Thats very nice! :blush:

4

Sounds great! So I guess something like these rules would work now?

https://github.com/CHEF-KOCH/dnscrypt-proxy-blacklist-filter

5

Yes (let's say "in principle"). The rules that are listed there aren't written in ERE, however, porting them over isn't complicated. However, note that to this point, it is entirely unexplored how adding a very long and complex regex would affect FTL's performance.

6

Thanks, just trying to understand. :slight_smile:

I'm sure lists like these adapted for the Pi-Hole will turn up soon. You can't subscribe to them like the usual lists though right?

7

No, for now you can specify one single regex in FTL's config. However, if the performance impact turns out to be negligible (we actually have no clue yet!) we might be able to extend this to a few regex strings. Note that, unlike "normal" blocking entries, regex can be really really powerful so it may be possible to combine much into only a very few expressions. We're eager to see what happens.

1 Like
8

4 posts were split to a new topic: Problems after checking out regex branch

10

To confirm, we can only have one line like that in the file? I had made a bunch with a different regex on every line.

11

I'll double check with @DL6ER but I think for now it is just one regex that is read. There were some concerns with multiple regexes and efficiency/system load but they may not be as much an impact as we were worried about.

12

Tested blocking via regex using
BLOCKINGREGEX=proxy|vpn|bypass|unblock|hide
The aim is to filter VPNs/proxy websites. I had previously used a hosts list of about 400 domains to achieve the blocking. However, blocking via the regex expression above makes about 80% of the entries on that list redundant and it even blocks random proxy sites found on google that I had not yet added to the block list. End result: Improved blocking, less work!
Thanks guys, love it! Keep up the good work!

13

Yes, the current version of FTLDNS in the beta testing supports only one regex. However, I already have a pull request open that will add support for an arbitrary number of regex filters (one per line in a new file /etc/pihole/regex.list) that needs to be reviewed by the dev team:

4 Likes
14

I considered merging all of my regex into one massive line, but with this on the horizon I am going to be patient.

Thank you.

1 Like