Pihole blocks Advertisements but not websites that are in the blocklists blocklist

Expected Behaviour:

Pihole should block URL's that are listed inside the Blocklists.

Actual Behaviour:

Websites that are supposed to be blocked by the Blocklist entrys of lists I added to the Blocklists are still reachable. However, if I add them manually to the Blocklist menue where I can do things with regex and wildcard, I sometimes cannot access them. All in all I can still acess them. I have tried adding xvideos.com and it is BOTH the most permitted and most blocked website https://i.imgur.com/M1py1Tp.png.

Debug Token:

uuejqdvzlw newer one: sy4c3oqrbf

I don't know what to do It's working on my Android device. It does not work on laptop and notebook. It seems to be blocking advertisements on windows but it isn't blocking me from visiting websites I blocked.
Did these things so far:

  • Have enabled the DHCP thing
  • DHCP IPv6 thing is also enabled
  • DCHP is disabled on my fritzbox
  • It is listed as DNS server in all fritzbox settings
  • I have rebooted all devices dozens of times
  • It does work for an URL I have added to the blacklist manually even on windows but it doesn't for the others I added manually
  • It did reinstall pihole
  • I added some URL's manually with the Blocklist menue where you can use wildcards. One of them is now being blocked. Others are not
  • Cleared Browsercaches and Sessions (everything but history)
  • Used /flushdns
  • Deactivated DoH in Firefox
  • Deactivated dns cache in Firefox
    ipconfig/all screenshot: https://i.imgur.com/0lKIQc0.png
    New Token: sy4c3oqrbf
    pihole -d: This process collects information from your Pi-hole, and optionally u - Pastebin.com

EDITS: Better English, new things I did, Screenshots.

It does!

xvideos.com is different from www.xvideos.com. Solution: In the Pi-hole Admin Console go to Blacklist, enter xvideos.com and click the "Add (wildcard)" button.

How can I automate that process? I'd have to insert more than a million URLs from hand for this to work. I have tried editing the wildcard list but there is noe. The URLs I have added to the wildcards are not listed at /etc/dnsmasq.d/03-pihole-wildcard.conf , Someone has mentioned that there is a script for this process. Do you know it's name? Where it is or even how it works. I can't find it.

I strongly doubt that. The blocklists/hosts files normally contain the domains/subdomains that should be blocked (barring exceptions, errors, omissions). If you add all domains in those lists as wildcards the probable outcome would be that far too much would be blocked.

EDIT: Sorry, I forgot to mention that the wildcards can be found in /etc/pihole/regex.list

What do you mean by too much? It's not like I browse websites with URLs of advertisements. Maingoal is to block all pornsites,if it starts blocking some clickbait website like top 10 funny pornconversations or something I wouldn't mind that.

Eitherway though? How do I add a whole list as wildcard? I found the regex.list but I don't know how to make a program write (^|.) infront of all of them.

Okay, it really depends what exactly you want to add as wildcard. If you want to add pornsites that's okay. What I meant: Adding a rule like, e.g., (^|.)spiegel.de$ just to block c.spiegel.de or count.spiegel.de would only cause problems.

That said, on https://firebog.net/ you'll find comprehensive lists that block pornsites.

You would have to write a little script using sed or awk. There are scripts by @anon55913113 and @mmotti in this forum which might be useful as a template.

I added all of those lists but they don't save pornsites like pornhub.com as www.pornhub.com making the website still accesible with www.pornhub.com and once I have reached them that way reachable without the www.

www.pornhub.com is in https://hostsfile.org/Downloads/hosts.txt which is also available on https://firebog.net/.

Besides, adding pornhub.com as a wildcard wouldn't block cases like
91pornhub.com
balkanpornhub.com
bitpornhub.com
bunkxxxpornhub.com
cryptopornhub.com
digitalpornhub.com
freexxxpornhub.com
httpspornhub.com
omegafreepornhub.com
periscopepornhub.com
pinaypornhub.com
ptpornhub.com
threesomepornhub.com
uspornhub.com
www.91pornhub.com
www.ar-pornhub.com
www.arvr-pornhub.com
www.arvrpornhub.com
www.balkanpornhub.com
www.bitpornhub.com
www.bunkxxxpornhub.com
www.cryptopornhub.com
www.digitalpornhub.com

So other lists are useful which cover those cases like https://hosts-file.net/psh.txt or a modified regex.

I saw somewhere that that you once made a script to do this. Is it still online? Can you post the topic where you shared it? I cannot find it :frowning:

Also what is a modified regex? Do you mean adding regex commands? Do they even apply to my list-blocked-entries? Don't I have to add every website that is supposed to be affected by the regex commands per hand? I did add some regex commands I found in a list already however.

Or just

pornhub\.

should be sufficient and also matches if the TLD changes.

If you add a domain like pornhub.com to the Blacklist in the Admin Console and click "Add (wildcard)" you'll get the following regex:

(^|\.)pornhub\.com$

This regex covers all subdomains of pornhub.com but not the examples mentioned in my post above. Hence, you could enter, e.g.,

pornhub\.

and click "Add (regex)" . This would match all those cases.

If you want to create regexes I suggest that you thorougly read Redirecting... . And as @anon55913113 mentioned, you can test your regexes on https://regex101.com/. A very helpful site is also https://regexper.com/.

Be careful, because this site uses more permissive rules than Pi-Hole. Pi-Hole uses POSIX ERE, and we have seen instances of regex that "pass" on regex101 and don't work in Pi-Hole.

So, even if the regex passes on regex101, you should enable regex debug mode in FTL and test a case on your Pi and see if it is blocked (each time a regex blocks, it becomes an entry in pihole-FTL log with debug enabled).

1 Like

Thanks - good to know! Do you know another site that should be used instead?

No, regex 101 is a good site, but here is some reading on ERE:

1 Like

I found one site which explicitly supports POSIX ERE as an option:
https://www.regexplanet.com/advanced/golang/index.html#

Unfortunately that site does not guarantee Pi-hole compatibility, either.

For example, from https://github.com/cbuijs/accomplist/blob/master/chris/regex.black I tried to add

^(.*\.)*(adult|porn|sex|sexy|xxx)$

to Pi-hole but got an error ("Wrong token") although it was accepted on that test site.

EDIT: I added

\.(adult|porn|sex|sexy|xxx)$

instead but that was not my point.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.