REGEX Domain Blacklist Issues

[jlw52761]

I am running Pi-Hole 5.3.1 on Ubuntu 20.04. Fresh setup and trying to get something working. I want to start by saying I did RTFM and verified that the REGEX I'm using is supported by FTL as everything's listed on that page.

I am trying to block those annoying YouTube ads, mainly as a challenge and to test Pi-Hole, and I have the following REGEX setup

(r\d+)((\-){3}|(\.))(sn)-(\S{8})(\S{4})?(\.googlevideo\.com)

Here's a list of what should be blocked (all except last one), but I'm seeing them as being passed on the logs

r4---sn-vgqs7nez.googlevideo.com
r20---sn-vgqs7nez.googlevideo.com
r7.sn-vgqs7ne7.googlevideo.com
r4.sn-8xgp1vo-xfgk.googlevideo.com
r23---sn-8xgp1vo-xfgk.googlevideo.com
r1---sn-vgqskned.googlevideo.com
r5---sn-vgqsrnll.googlevideo.com
r3---sn-vgqsknll.googlevideo.com
r5---sn-vgqsrnll.googlevideo.com
r1---sn-cxoxnug5-cune.googlevideo.com

In using pihole-FTL --regex-test, I get this

pihole-FTL regex-test "r3---sn-vgqsknll.googlevideo.com" '(r\d+)((\-\-\-)|(\.))(sn)-(\S{8})(\S{4})?(\.googlevideo\.com)'
[i] Compiling regex filter...
    Compiled regex filter in 0.282 msec

Checking domain...
    (r\d+)((\-\-\-)|(\.))(sn)-(\S{8})(\S{4})?(\.googlevideo\.com) matches
   Time: 0.434 msec

It's matching there, but not anywhere else. I have done the restart of FTL using

sudo service pihole-FTL restart

I have also restarted the DNS service and restarted the entire machine. I'm confused. BTW, the list of URLs I pulled from the logs.

Help

[jfb]

@jlw52761

the list of URLs I pulled from the logs.

Domains, not URLs.

Please upload a debug log and post just the token that is generated after the log is uploaded by running the following command from the Pi-hole host terminal:

pihole -d

or do it through the Web interface:

Tools > Generate Debug Log

[yubiuser]

@jlw52761

I am trying to block those annoying YouTube ads,

A lot have tried, all failed in the end. There is no way to block Youtube ads without blocking video content as well.

How do I block ads on YouTube?

Update 2020-10-24 It's not likely that you'll be able to block in-video ads with Pi-hole. For that, you'll need something like uBlock origin for desktop, or vanced for android (Now defunct), This thread has been closed as there is not much more discussion to be had around the topic. Update 2018-11-14 User's have had good success here: I'm updating this top-level post as of 2018-10-16 since a lot of people land here. Blocking ads on YouTube is hit or miss with Pi-hole. The reason for th…

[jlw52761]

I have to wonder if this issue I'm looking at has been the problem all along...

[jlw52761]

The debug token is https://tricorder.pi-hole.net/s4phuxtpz2

[Bucking_Horn]

@jlw52761

I have to wonder if this issue I'm looking at has been the problem all along...

Are you speculating that solving your issue would pave the way for blocking youtube ads by DNS filtering? Don't speculate further - it won't.
You may perhaps see a reduction of ads for a time, until the domain patterns get changed again by Google.

---

By default, clients will be filtered by the Default group, and that group only.

Your debug token shows you have assigned your regex to a group Custom_Lists with group id 3.
However, there are no clients that make use of that group.

*** [ DIAGNOSING ]: Groups
   id    enabled  name                                                date_added           date_modified        description                                       
   ----  -------  --------------------------------------------------  -------------------  -------------------  --------------------------------------------------
   3           1  Custom_Lists                                        2021-06-03 22:36:09  2021-06-03 22:36:09  My Custom Lists                                   

*** [ DIAGNOSING ]: Domainlist (0/1 = exact white-/blacklist, 2/3 = regex white-/blacklist)
   id    type  enabled  group_ids     domain                                                                                                date_added           date_modified        comment                                           
   ----  ----  -------  ------------  ----------------------------------------------------------------------------------------------------  -------------------  -------------------  --------------------------------------------------
   8        3        1  3             (r\d+)((\-){3}|(\.))(sn)-(\w+)(-\s+)?(\.googlevideo\.com)                                             2021-06-03 23:55:00  2021-06-03 23:57:10  Youtube Ads                                       

*** [ DIAGNOSING ]: Clients

Either use Client group management to define clients that you want to apply your Custom_Lists to and assign that group to them, or use Domain management to move all of your Custom_Lists content to the Default group.

[jlw52761]

I'm not speculating, I see that the regex based blocking is not working. If it doesn't block YouTube, so be it, as long as we can show the regex based blocking will work.

[Bucking_Horn]

@jlw52761

I'm not speculating

Then I may have misunderstood your reply to yubiuser's remark on the general futility of blocking Youtube ads by DNS means:

@jlw52761

I have to wonder if this issue I'm looking at has been the problem all along...

When reading through that long-running topic yubiuser has linked to, you will find that each blocking attempt has been thwarted sooner or later.
Browser extensions have a better chance to block those ads, as they are aware of the full URL, not just domains.

@jlw52761

I see that the regex based blocking is not working.

It's working, but with no clients using that group, you won't see any clients getting filtered by that regex.
The steps mentioned in my previous post above should fix that.

[jlw52761]

I'm not sure I would say it's working

And looking at this, I would expect to see that it's blocked, not passed:

pihole-FTL regex-test 'r4---sn-vgqsknes.googlevideo.com'
[i] Loading regex filters from database...
    Compiled 1 black- and 2 whitelist regex filters in 1.990 msec

[i] Checking domain against blacklist...
    (r\d+)((\-){3}|(\.))(sn)-(\w+)(-\s+)?(\.googlevideo\.com) matches (regex blacklist, DB ID 8)
    Time: 0.282 msec
[i] Checking domain against whitelist...
    Time: 0.006 msec

[Bucking_Horn]

Which change did you apply?

We need a new debug token, together with the information which client requested resolution for r4---sn-vgqsknes.googlevideo.com.

[jlw52761]

https://tricorder.pi-hole.net/a9aiao3p42

Client is Jody-iPhone.dhcp.mus.tiberiansun.us

[Bucking_Horn]

Your debug log suggests you didn't change anything?

EDIT:

*** [ DIAGNOSING ]: Groups
   id    enabled  name                                                date_added           date_modified        description                                       
   ----  -------  --------------------------------------------------  -------------------  -------------------  --------------------------------------------------
   3           1  Custom_Lists                                        2021-06-03 22:36:09  2021-06-03 22:36:09  My Custom Lists                                   

*** [ DIAGNOSING ]: Domainlist (0/1 = exact white-/blacklist, 2/3 = regex white-/blacklist)
   id    type  enabled  group_ids     domain                                                                                                date_added           date_modified        comment                                           
   ----  ----  -------  ------------  ----------------------------------------------------------------------------------------------------  -------------------  -------------------  --------------------------------------------------
   8        3        1  3             (r\d+)((\-){3}|(\.))(sn)-(\w+)(-\s+)?(\.googlevideo\.com)                                             2021-06-03 23:55:00  2021-06-03 23:57:10  Youtube Ads                                       

*** [ DIAGNOSING ]: Clients

[jlw52761]

@Bucking_Horn

2021-06-03 22:36:09

Looks like the right timestamp on when I last updated the REGEX rule. I haven't touched the rule since last night.

[Bucking_Horn]

Please decide on one of the changes I suggested and apply it.

[jlw52761]

I'm confused on either of those options on what would need to be done, and what implications are of each. I have a few entries in the Domain Management section, two of which are domain wildcard for whitlist, and then this REGEX. I have nothing under Clients.

[Bucking_Horn]

The easiest approach would be navigate to the Domain management RegEx filter tab and change the Group assignment for your regex to Default.

For client-specific blocking, have a read of Pi-hole's Per-client blocking example

[jlw52761]

Ok, interesting, it is now matching that REGEX pattern. So really, the Groups can't be used with REGEX Domains then, is what I'm gathering.

[Bucking_Horn]

@jlw52761

So really, the Groups can't be used with REGEX Domains then, is what I'm gathering.

I think you misunderstand Pi-hole's groups:
Groups are designed to facilitate client specific blocking.

Any RegEx assigned to a group will take effect as soon as you assign respective clients for that same group.

If you do not or cannot use client-specific blocking, there is no use in setting up groups. Just assign all your definitions to the Default group then (which is what will happen by default).

[jlw52761]

Ok, yes, I did misunderstand them. I looked at Groups like a method of grouping together feeds. Thank you for the clarification.