Using BlockList to add wildcard blocked domains?


#1

Hello all I am trying to add Blacklisted wildcard domains using the Block List section.
This would be nice so that I don’t have to add them all by hand I can just use a txt file hosted on my web server to add them all in one shot.

Here is part of the full list of domains that I have.
I would like to just use a one line wildcard in the txt file. Is that possible ?

lunrac.com
www.lunrac.com
uk0.lunrac.com
uk1.lunrac.com
uk2.lunrac.com
uk3.lunrac.com
uk4.lunrac.com
uk5.lunrac.com
uk6.lunrac.com
uk7.lunrac.com
uk8.lunrac.com
uk9.lunrac.com
uk10.lunrac.com
uk11.lunrac.com
uk12.lunrac.com


#2

Add wildcard lunrac.com under blacklist.


#3

Yes I am aware that is a way of doing it.

But that is not what I am asking in this post.


#4

I may not understand what you are trying to do.

Block lists are web sites that contain a text file of domains to be added to the gravity list.

Blacklists are local lists that your Pi-Hole processes (/etc/pihole/blacklist.txt).

In Pi-Hole V4, wildcards are converted to regex filters and stored in /etc/hosts/regex.list.

Which .txt file do you wish to add a one-line wildcard to?


#5

I have a locally hosted .txt file with a bunch of domains that I have added to the Gravity listing.
I would like to thin the list of domains by using an wildcard domain in the list instead of 30+ subdomains.xyz.com

How it is now:
1sub.xyz.com
2sub.xyz.com
3sub.xyz.com
4sub.xyz.com

It could be:
*.xyz.com

Here is my personal list of blacklisted domains : https://sesipod.info/DATA_FILES/pfsense/personalblock.txt


#6

The problem with this method is that when gravity is rebuilt (either manually or via the weekly cron script), your local changes can be overwritten. If you are adding these domains, they should be added to the blacklist, not directly to the gravity list.

The best way to incorporate your lists is to set up regex filters. Documentation here - there is tutorial link in the left column on that page.

https://docs.pi-hole.net/ftldns/regex/overview/

In the web GUI (blacklist), add your domain as a wildcard, and Pi-Hole will convert that to a regex expression for you and add it to the regex list.

Example:

Add wildcard “lunrac.com” and this becomes regex (^|.)lunrac.com$. It would look like this in a regex diagrammer (https://regexper.com).

image


#7

So are you saying that the List’s that I have added to the Gravity listing will be removed?


#8

No. I misunderstood how you were doing this. The way you have done it is correct. When gravity is built or updated, it will read your list and include your domains in gravity.

You can check this by searching the gravity list for one of your domains.

pihole -q , or through the web interface, or with a grep

sudo grep lunar /etc/pihole/gravity.list

But to your original question, you can put your list domains in regex format, and if that covers them all, then delete your local list from your block lists.


#9

No worries :smiley:

Okay so now that we are on the same page. Is there a way to add the wildcard domains to my .txt file that I have added to the Gravity lists?


#10

I don’t believe so. The file gravity.list is simply a list of domains, without wildcards.

You have essentially two choices:

  1. Leave your list as is and have a few extra lines in the gravity.list. That won’t affect performance, as gravity is searched very quickly during a DNS request. Even if you had 1,000 domains on your personal list, that is minute in comparison to the 130,000 domains on the seven standard block lists.

  2. Put your list items into regex format, then remove your list from the block lists, or the domains will end up in multiple places (block list and regex list).


#11

Is there a way to make a backup of this method for deployment to a second pi-hole DNS server? Is the list stored somewhere that is accessible via sftp on the Linux server? If I where to move it to the other box would Pi-Hole pickup on the changes to the list.


#12

The lists are all in folder /etc/pihole.

The teleporter feature (web GUI > settings > teleporter) exports all the lists in an archive. You can save this to a location, then when the new Pi-Hole is up and running, import them from the network location.

Contents of a teleporter archive:

image


#13

It seems like its all stored under gravity.list :thinking:


#14

AH! Found it … regex.list under /etc/pihole 213


#15

So I just manually edited the regex.list under /etc/pihole and it auto updated the list on the web GUI so I guess this will work :stuck_out_tongue: :man_shrugging:


#16

You don’t want these regex items in gravity. Gravity works on HOSTS format (single domain per line).


#18

So I checked the lists that I made and they are not added there so I remove them form the Gravity list using NANO and then re-updated the Gravity listing and it added them back :0 ?

Here are my lists…
https://sesipod.info/DATA_FILES/pfsense/blockytube.txt
https://sesipod.info/DATA_FILES/pfsense/personalblock.txt


Update —

I fixed the issue by removing my lists running an update then adding my lists back and updating. I guess it was caching the list.


#19

Do you have success by using those blocks for Youtube?


#20
I do have some success with blocking ads here is my full list.

https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://mirror1.malwaredomains.com/files/justdomains
http://sysctl.org/cameleon/hosts
https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
https://hosts-file.net/ad_servers.txt
https://adblock.mahakala.is/
https://someonewhocares.org/hosts/hosts
http://winhelp2002.mvps.org/hosts.txt
https://www.squidblacklist.org/downloads/dg-ads.acl
https://pgl.yoyo.org/adservers/serverlist.php?hostformat=nohtml
https://sesipod.info/DATA_FILES/pfsense/blockytube.txt
https://sesipod.info/DATA_FILES/pfsense/personalblock.txt

#21

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.