Only allow Playstation domains, while blocking Playstation tracking domains at the same time

[spicyblocker]

Expected Behaviour:

Scenario:
My Playstation 5 is part of the groups "Playstation" and the default group.

• I have a regex blacklist of . for Playstation. This means no domains are allowed by default. This is because I want to use the Playstation purely as a video game console and don't want video thumbnails, streaming apps etc. being pushed down my throat.
• I have a regex whitelist of playstation.com and playstation.net for the group Playstation
• Some subdomains (like telemetry-console.api.playstation.com and smetrics.aem.playstation.com) are ad domains which are present in adlists that are assigned to the default group. They have also been added as exact blacklisted domains to the Playstation group.

I would like the ad/tracking domains to be blocked.

Actual Behaviour:

This is not the case. From looking around the forum, I found that Pi-Hole prioritizes whitelisting, which makes sense for most scenarios:

Exact Whitelist
Regex Whitelist
Exact Blacklist
Blocklist domains (AKA gravity)
Regex Blacklist

Therefore, I assume I need to take a look at rewriting the whitelist regex so it excludes the two bad domains. I looked at the regex tutorial but still can't figure it out. Would be grateful if someone could ELI5 this to me

[rdwebdesign]

Please upload a debug log and post just the token URL that is generated after the log is uploaded by running the following command from the Pi-hole host terminal:

pihole -d

or do it through the Web interface:

Tools > Generate Debug Log

[spicyblocker]

Sure, here you go!

[rdwebdesign]

Before start to "debug" I have a question.

You used "domain" and "specificsub.domain" instead of real domains. This makes me think you want, for some reason, to hide the actual domains.

This will make harder to explain and find out the real issue. If you don't mind to show the actual domains in your list, it will be simpler.

Can I post the real domains and group names used by you?

[spicyblocker]

Yes, feel free to name the domain and group names. The domains should be

• playstation.com
• playstation.net

for the regex whitelists and

• smetrics.aem.playstation.com
• telemetry-console.api.playstation.com

for the exact blacklisted domains, which should also be in one or more adlists.

I wanted to keep it generic because I thought this was a problem of Pi-Hole prioritizing whitelisted domains, hoping it would be fixed by changing the regex whitelist rule.
But you're implying another cause here, so I think others would benefit from this thread if the context is clear. I'll change the title of the thread if it becomes clear this is an issue specific to these domains and/or the Playstation.

[rdwebdesign]

You created a regex to block everything: (.) - it will match any character. So any domain containing at least one character will match and will be blocked.

You also created a rule to allow a specific domain: playstation.com - this will allow any domains containing playstation, followed by any character, followed by com.
Examples:

• playstation.com
• playstation4com.org
• smetrics.aem.playstation.com
  This will allow the subdomains you want to block (See here - every line has a match).

   276       3        1  1             .                                       <--- Regex to block everything in group 1 
   (...)
   287    0           1  0             www.playstation.com                     <--- exact domain whitelist in group 0 (but not in group 1)   
   (...)
   302      2         1  0             communication.playstation.net           <
   (...)
   309    0           1  0             status.playstation.com                  <--- exact domain whitelist in group 0 (but not in group 1) 
   (...)
   317      2         1  1             playstation.net                         <--- Regex to allow, in group 1
   318      2         1  1             playstation.com                         <--- Regex to allow, in group 1 
   319     1          1  1             smetrics.aem.playstation.com            <--- (ignored) It won't block because rule 318 will allow it 
   320     1          1  1             telemetry-console.api.playstation.com   <--- (ignored) It won't block because rule 318 will allow it

You can change your allow rule to ^playstation\.com$. This will allow only the playstation.com domain exactly, but no subdomains (like this - only the first line will match).

To be honest, I think a more simple way to do it is to remove the . block regex and only block the domains and subdomains you want to block.

---

Another thing:

You created a group called Playstation (ID=1) and added some block rules to this group (including the .), but you added some exact whitelist domains (like store.playstation.com) to the default group (ID=0).

Depending on your rule, this will cause issues if you include a device in group 1, but not in group 0.

[spicyblocker]

Wow, thanks for this very detailed explanation!

I added exact whitelisted domains like store.playstation.com before deciding that there were simply too many playstation domains to whitelist them all. Forgot to remove them after adding the regex playstation.com.

My rationale behind the . regex blacklist is I don't like how the Playstation essentially became a FireTV stick thing, with ads, video thumbnails, streaming apps etc. being pushed down your throat.

Reading your response, I thought this is a scenario which can't be covered with regex rules.

I tried asking the friendly neighboordhood LLM just in case and here's what I got.

^(?!telemetry-console\.api\.playstation\.com$)(?!smetrics\.aem\.playstation\.com$)([a-z0-9-]+\.)*playstation\.com$

Pi-Hole happily accepts this and it seemed to work.

I then noticed that certain domains would not be covered by the regex whitelist, such as:

• feature.api.playstation.com
• takedown.api.playstation.com

These were then caught by the . regex blacklist and blocked. After a bit of back and forth with the LLM, I got this:

/^(?!telemetry-console\.api\.playstation\.com$)(?!smetrics\.aem\.playstation\.com$)([a-z0-9-]+\.){0,}[a-z0-9-]+\.playstation\.com$|^playstation\.com$

If you follow both links, you'll see that in both proposed regex rules, domains like feature.api.playstation.com are actually included. However, Pi-Hole still blocks them. When I click on the reason (see picture), it links to the . regex blacklist.

This could mean that Pi-Hole interprets the regex whitelist differently than regex101.com. For some reason, it's not including any subdomain of api.playstation.com. Checking Pi-Hole diagnosis, I see:

Is this intended?

For the time being, I'll go with the first LLM-proposed regex rule and try to whitelist all the api.playstation.com subdomains. There only seem to be a handful (for now at least).

Also, after seeing this flaw in the regex rule:

You also created a rule to allow a specific domain: playstation.com - this will allow any domains containing playstation, followed by any character, followed by com.
Examples:

• playstation.com
• playstation4com.org
• smetrics.aem.playstation.com

...I rewrote the playstation.net regex whitelist to:
^([a-z0-9-]+\.)*playstation\.net$

[rdwebdesign]

The Pi-hole regex accepts only a specific set of regex rules. It is fast, but it doesn't accept some grouping rules (like you used above).

From Pi-hole docs:

We implement the POSIX Extended Regular Expressions similar to the one used by the UNIX egrep (or grep -E) command. We amend the regex engine by approximate blocking (compare to agrep) and other special features like matching to specific query types only.

This makes very hard to block everything, then "unblock" a domain, but not its subdomains.

I still think it would be easier if you only block the undesired domains.
There will be many domains to block (probably just as many as you currently want to unblock), but it will be easier to find them and block if you look at the query log.

[spicyblocker]

Ah... interesting.

In my scenario, I'd rather go for overblocking than risking underblocking. But that's just because I don't get annoyed that easily

But I imagine there's others wishing for an implementation of this logic. I'm sure this is not the first time someone has wanted to only allow whitelisted (sub)domains while not allowing some subdomains of the allowed domain.

Making changes to the type of regex implemented by FTL sounds like a lot of work.
But what if we added another type of blacklist to get around it?

For example, instead of:

Exact Whitelist
Regex Whitelist
Exact Blacklist
Blocklist domains (AKA gravity)
Regex Blacklist

We could have:

Exact Whitelist
"Special" Exact Blacklist
Regex Whitelist
Exact Blacklist
Blocklist domains (AKA gravity)
Regex Blacklist

Is addressing this something the team would consider?

[rdwebdesign]

No... Whitelists will always "win".

Pi-hole search for whitelist rules first. If the domains matches with a whitelist, it will be allowed and no blocklist rules will be tested.

In addition to Blocklists, Pi-hole v6 will accept Allowlists: