CNAME block list

The developpers have added a great feature to pihole v5 called 'Deep CNAME inspection', you can read about it here.

The block list maintainers will have to add entries for the CNAMEs, in order for this to feature to activelly block domains.

Found the first reference](GitHub - nextdns/cname-cloaking-blocklist: A list of domains used by tracking companies as CNAME destination when disguising third-party trackers as first-party trackers.) that actually addresses CNAME entries (unfortunately it's already two months old). There are some links to reading material to explain in further detail why you need CNAME blocking.

As explained in the beta5 announcement, you can disable 'Deep CNAME inspection', using the setting CNAME_DEEP_INSPECT=false in /etc/pihole/pihole-FTL.conf

There is a warning on the home page that says:

Important: For this list to work at all, the blocking logic needs to wildcard match (domain and all its multi-level subdomains) CNAMEs against the domains in this list.

The question for the developpers:
what is the recommended way to add these CNAME domains?
will adding a dnsmasq (separate config file) entry (random entry from the reffered list) such as

address=/wizaly.com/#

trigger CNAME blocking OR does it require a regex, such as

(\.|^)wizaly\.com$

edit
tried to find the answer myself. apparently, using dnsmasq wildcard syntax (address=/wizaly.com/#) does NOT trigger CNAME detection, regex syntax does trigger CNAME detection.

Still would like confirmation this is the best possible regex to comply to the requirements for CNAME cloaking, usin the entries from the NextDNS article.
/edit

There's no magic to "trigger" anything. If a domain query is a CNAME then that CNAME is checked for white/black/block. All domains found in the chain from queried domain to final IP are checked.

1 Like

Dan is absolutely right here.

Yes, this is expected. If you specify it using a dnsmasq-instruction, you effectively add the domain wizaly.com to dnsmasq's DNS cache. If you query this domain (or any of its subdomains) directly, you will get the reply from the cache. During a CNAME traversal, however, dnsmasq does not look at its cache at all (at least not at if there are parts of it already known).

Black- and whitelists (may it be regex or exact) are an FTL feature and are always checked and are to be used here.

Is this then a correct entry to block all possible CNAME entries for wizaly.com? Or do you recommend something else to achieve blocking the CNAME domains, referred here? I'm asking, since we are all new to CNAME blocking, and probably want to use an entry, matching what the developers intended.

Yes, FTL does a lot to have internal CNAME bits look like regular domains to the users so known rules can be applied to them.

Wildcard block wizaly.com. If it shows up in a direct query, it will be blocked. If it shows up as a CNAME for a different domain query, it will be blocked. There really isn't any behavior to change for CNAMEs, block what you want blocked and let FTL handle the internals.

I hoped so, been busy all afternoon to get this script working, see here. If you have any comments, feel free to shoot. The goal of the script is to read the domains from the file and enter them directly into the database. Tested, works, but comments are expected.

I wrote this, because pihole has no facilities to achieve this.

Strange.

You should let FTL know about the incoming new regex filters as it needs to compile them for you before they can be used. Add

pihole restartdns reload-lists

at the end of your script. This should do everything you need.

thanks for the comment (strange?)

I will add the reload command to the script.... done.

Yeah, strange as in: I would not have expected it to work.
You script only updated the database but did not made FTL check for new/deleted regex.

As FTL does not compile (= load) the regex at the end of your script. So it probably worked only by chance because you're restarted FTL at some point or did some changes to the lists using either the CLI or the web interface (both will run the reload-lists for you).

1 Like

I am reasonably good in breaking down a problem into the steps, needed to solve the problem, terrible when turning the steps into code (trial and error, using duckduckgo examples to translate a step into code).
This sometimes ends up in simply rebooting, due to a watchdog trigger (max-load-15), or manual intervention if the system is still responsive.
I'm using this to look at the database, but it's a lousy way to apply massive changes (remove all regex entries to avoid falsifying the test result).
I also tried to remove all blocklists (except one) from the database, but failed (timeout). It even looks like you cannot remove a blocklist from the database, only enable/disable it (can you confirm or deny this?).
Anyway, I rebooted, that caused FTL to read (compile).

Deny. Enabling/disabling is writing to the database as much as deleting is.

4 posts were split to a new topic: Discussion about optimal wildcard syntax

Interesting article on reddit.
One comment says:

Those domains (and many others) all seem to CNAME to ghochv3eng.trafficmanager.net (see https://securitytrails.com/list/cname/ghochv3eng.trafficmanager.net).
We have added it to our CNAME cloaking blocklist (that can be used with Pi-hole):
https://github.com/nextdns/cname-cloaking-blocklist

This comment refers to the blocklist discussed here.

A script, I'm running weekly (cron) to add possible new entries (one this week) to pihole (regex blacklist) can be found here

result, for the new entry, referred in the reddit article:

Thanks, works like a charm!

Does the script throw an error for you if there has been no change to the list?

I get the following output:

--2020-04-28 13:55:49--  https://raw.githubusercontent.com/nextdns/cname-cloaking-blocklist/master/domains
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.196.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.196.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1011 [text/plain]
Saving to: ‘/home/pi/domains’

/home/pi/domains                                  100%[===========================================================================================================>]    1011  --.-KB/s    in 0.001s  

2020-04-28 13:55:49 (1.00 MB/s) - ‘/home/pi/domains’ saved [1011/1011]

nextdnstrack.sh: 12: nextdnstrack.sh: Syntax error: "(" unexpected (expecting "fi")

No it doesn't. You are probably experiencing the copy/paste problems, related to discourse, which changes certain characters, making the copied script fail. You can download the script here.

Turns out I am an idiot. I was running it with sh nextdnstrack.sh instead of ./nextdnstrack.sh.

Using code fence language identifiers should stop that from happening. No smartquotes.


    ```bash
       Shell code goes "$here"
    ```

And full code blocks should be copyable with the copy icon on hover.

Hi. Thanks for the script. I copied it over (downloaded it from you).
But when I run

$ sh /usr/local/bin/NextDNS.sh
or
$ sudo sh ./usr/local/bin/NextDNS.sh

I get always the same error as mentioned above:

--2020-05-16 19:21:57--  https://raw.githubusercontent.com/nextdns/cname-cloaking-blocklist/master/domains
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.36.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.36.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1011 [text/plain]
Saving to: ‘/home/pi/domains’

/home/pi/domains             100%[=============================================>]    1011  --.-KB/s    in 0s

2020-05-16 19:21:57 (6.85 MB/s) - ‘/home/pi/domains’ saved [1011/1011]

./usr/local/bin/NextDNS.sh: 11: ./usr/local/bin/NextDNS.sh: Syntax error: "(" unexpected (expecting "fi")

Any idea why?
Thank you in advance!