Inconsistent CNAME blocking?

General Beta 5.0
1

What browser do you use, apparently Edge is regularly trying to connect to www.msn.com. That's why I have so many hits on the domain.

In my config (using a lot of lists) pihole reports:

pihole -q www.msn.com
  [i] No results found for www.msn.com within the block lists

but the CNAME is on my personal list

pihole -q a-0003.a-msedge.net
 Match found in http://localhost/mylist.txt:
   a-0003.a-msedge.net

You can create a local list (your personal entries) by:
creating a file in /var/www/html/mylist.txt and adding the list http://localhost/mylist.txt
OR
creating a file in /home/pi/mylist.txt and add the list file:///home/pi/mylist.txt

2

Well, maybe not. @jpgpi250 is still seeing issue(s) so I'm not yet done. Don't misunderstand me, I was busy this weekend. I do appreciate @jpgpi250's tries to "destroy" Pi-hole using an uncommon configuration. A properly working code should be fully deterministic so there is still at least a cache hiccup which we should resolve before sending this code towards release/v5.0.

I'd need more line, also from some blocked queries so I can investigate the difference. I have no Windows Pcs available at home and, strangely enough, failed to reproduce your issue previously with manually triggered queries for www.msn.com.

8

Sure thing, no bug is allowed to survive!

9

I've run some database queries for the domain ctldl.windowsupdate.com.
You have a copy of my database (see PM), so you can run the same queries on a (renamed) database file.
There are 3 windows machines (2x win10, 1xwin7) in use. On al the devices, the OS apparently requests info for ctldl.windowsupdate.com. The problem (sometimes blocked, sometimes allowed) exists on all devices, so I assume it is not device related.

On the pihole-FTL.db, queries table, I ran the following database query:

select * from queries where domain="ctldl.windowsupdate.com";

The type field indicates both A (type 1) and AAAA (type 2) queries are registred.
The status field indicates two results, Permitted + forwarded (status 2) and unknown status 9 (The documentation isn't up to date yet). I assume this value indicates blocked due to CNAME. The status 9 entries always have NULL as the forward value, I assume this means the query is never forwarded to unbound and pihole-FTL returns 0.0.0.0.

FIRST CONCLUSION (but I may be wrong):
I assume the domain ctldl.windowsupdate.com always triggers CNAME detection, in fact I assume ALL domains trigger CNAME detection, if the domain passes the whitelist, blacklist, gravity and regex tests. Since the status is sometimes 2 (Permitted and forwarded) and sometimes 9 (blocked due to CNAME), This would mean the CNAME logic is NOT always comming up with the same result.

I've checked the dig results for the domain ctldl.windowsupdate.com, resolved using unbound, thus bypassing pihole-FTL. The result for the A and AAAA queries are different:

  • A query: audownload.windowsupdate.nsatc.net. 0 IN CNAME au.download.windowsupdate.com.hwcdn.net.
  • AAAA query: audownload.windowsupdate.nsatc.net. 0 IN CNAME au.download.windowsupdate.com.edgesuite.net.

however, the result is always an A or AAAA address, none of the domains are in a blocklist (I don't really know if pihole -q also checks the regex entries, to determine if a domain is blocked or allowed)

pi@raspberrypi:~ $ dig @fdaa:bbcc:ddee:2::5552 -p 5552 A ctldl.windowsupdate.com

; <<>> DiG 9.11.5-P4-5.1-Raspbian <<>> @fdaa:bbcc:ddee:2::5552 -p 5552 A ctldl.windowsupdate.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41440
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1472
;; QUESTION SECTION:
;ctldl.windowsupdate.com.       IN      A

;; ANSWER SECTION:
ctldl.windowsupdate.com. 2484   IN      CNAME   audownload.windowsupdate.nsatc.net.
audownload.windowsupdate.nsatc.net. 0 IN CNAME  au.download.windowsupdate.com.hwcdn.net.
au.download.windowsupdate.com.hwcdn.net. 2484 IN CNAME cds.d2s7q6s2.hwcdn.net.
cds.d2s7q6s2.hwcdn.net. 0       IN      A       205.185.216.42
cds.d2s7q6s2.hwcdn.net. 0       IN      A       205.185.216.10

;; Query time: 6 msec
;; SERVER: fdaa:bbcc:ddee:2::5552#5552(fdaa:bbcc:ddee:2::5552)
;; WHEN: Mon Feb 03 10:04:01 CET 2020
;; MSG SIZE  rcvd: 209

pi@raspberrypi:~ $ dig @fdaa:bbcc:ddee:2::5552 -p 5552 AAAA ctldl.windowsupdate.com

; <<>> DiG 9.11.5-P4-5.1-Raspbian <<>> @fdaa:bbcc:ddee:2::5552 -p 5552 AAAA ctldl.windowsupdate.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21451
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1472
;; QUESTION SECTION:
;ctldl.windowsupdate.com.       IN      AAAA

;; ANSWER SECTION:
ctldl.windowsupdate.com. 2463   IN      CNAME   audownload.windowsupdate.nsatc.net.
audownload.windowsupdate.nsatc.net. 0 IN CNAME  au.download.windowsupdate.com.edgesuite.net.
au.download.windowsupdate.com.edgesuite.net. 0 IN CNAME a767.dscg3.akamai.net.
a767.dscg3.akamai.net.  0       IN      AAAA    2a02:26f0:ed::5c7a:352b
a767.dscg3.akamai.net.  0       IN      AAAA    2a02:26f0:ed::5c7a:3512

;; Query time: 6 msec
;; SERVER: fdaa:bbcc:ddee:2::5552#5552(fdaa:bbcc:ddee:2::5552)
;; WHEN: Mon Feb 03 10:04:22 CET 2020
;; MSG SIZE  rcvd: 242

I than checked wheter any of the CNAME entries are on a blacklist:
For the A query, the result is:

pi@raspberrypi:~ $ pihole -q ctldl.windowsupdate.com
  [i] No results found for ctldl.windowsupdate.com within the block lists
pi@raspberrypi:~ $ pihole -q audownload.windowsupdate.nsatc.net
  [i] No results found for audownload.windowsupdate.nsatc.net within the block lists
pi@raspberrypi:~ $ pihole -q au.download.windowsupdate.com.hwcdn.net
  [i] No results found for au.download.windowsupdate.com.hwcdn.net within the block lists
pi@raspberrypi:~ $ pihole -q cds.d2s7q6s2.hwcdn.net
  [i] No results found for cds.d2s7q6s2.hwcdn.net within the block lists

For the AAAA query, the result is:

pi@raspberrypi:~ $ pihole -q ctldl.windowsupdate.com
  [i] No results found for ctldl.windowsupdate.com within the block lists
pi@raspberrypi:~ $ pihole -q audownload.windowsupdate.nsatc.net
  [i] No results found for audownload.windowsupdate.nsatc.net within the block lists
pi@raspberrypi:~ $ pihole -q au.download.windowsupdate.com.edgesuite.net
  [i] No results found for au.download.windowsupdate.com.edgesuite.net within the block lists
pi@raspberrypi:~ $ pihole -q a767.dscg3.akamai.net
  [i] No results found for a767.dscg3.akamai.net within the block lists

Since none of the CNAME entries are blocked, pihole-FTL should always permit the domain(s).

This is probably very hard to read, so I've send you a PM to allow you to download my files.
content:
pihole.log (todays pihole log)
pihole.log.1.gz (yesterdays pihole log, starting from 22h10)
pihole-FTL.log (todays pihole-FTL log DEBUG_QUERIES enabled)
pihole-FTL.log.1.gz (yesterdays pihole-FTL log DEBUG_QUERIES enabled, starting from 22h10)
pihole-FTL.db
cnames.csv (the result of select * from queries where status=9;)
pi-hole-teleporter_2020-02-03_12-24-38.tar.gz (teleporter export)
pihole_debug.log (included or https://tricorder.pi-hole.net/p3n7nibpfq)

There are a lot of entries in cnames.csv, however, a lot of duplicate domains, very few actual unique domains.
In order to check for a possible problem, I use the web interface / Long Term Data / query log, select a time frame and enter the domain name from the csv into the seach box. All green OR all red -> no problem. Mixed green and red -> that's the problem.

Really hopes this info assists to identify the problem.

10

By now, I know how much you hate any configuration, deviating from what the developers originally intended, which is buster on raspberry pi + pihole install + optionally unbound, nothing more.

Although I understand, from a point of providing support, the reason for this, there will always be users, wanting more sophisticated features, be it for the simple reason that it's possible.

I'm also very aware you don't like the redis solution, and don't want to support it.

To eliminate this discussion, I've done the following:

  • stopped unbound, redis and pihole-FTL
  • /etc/dnsmasq.d/01-pihole.conf, cache-size=10000
  • removed logs and pihole-FTL.db
  • removed redis config file from unbound (config is a separate file, by removing it unbound will no longer attempt to use it)
  • restarted unbound and pihole-FTL, did NOT restart redis (no longer in use), all stats in pihole dashboard are now zero.
  • rebooted two out of three windows machines.

Within minutes, I noticed the following:

image
image2333×356 14.2 KB

I know, you'll probably still be sceptic about my config, for me, this proves it has nothing to do with redis.

Please accept my apologies if I have somehow offended you, this is absolutely NOT my intention. Just want to assist to get a bug free version of pihole v5.

11

The CNAME-Thing is not a bug. It will happen every time when domains in a CNAME have to be blocked due to blacklisted names - no matter if own blacklist or gravity. I found more domains were CNAME_DEEP_INSPECT=true in the pihole-FTL.conf will get you to madness because you are nearly unable to resolve the real problem. Today I spent hours to find out wich entry at the regex list was keeping my network radio blocked. A combination of traceroute, browser and ping solved the problem. The Tail log of Pihole alone did not.
The new CNAME deep inspection is a great thing, but without help via the user interface that says what is really blocked nearly useless.
If there's just to much frustration because of it, temporaryly disable it with CNAME_DEEP_INSPECT=false at the pihole-FTL.conf file. Restart FTL then and the behavoir is just like the one you know from v4.3.2.

12

This topic (a lot of reading) was started by somebody who noticed A and AAAA records are sometimes treated differently (blocked or allowed). The developer created a new branch, outside beta5 to try to tackle this problem. Part of the solution for the problem is a modification to web interface, showing exactly that what is the problem (what is really blocked). The screenshot shows exactly that, you just will NOT YET see this in the beta5 branch, because the code hasn't been approved and merged.

Screenshot at 2020-01-29 22-57-40
Screenshot at 2020-01-29 22-57-40935×108 11.2 KB

Unfortunately, there is a difference between looking at queries in the query log menu (shows the CNAME) and looking at queries in the long-term data / guery log menu (doesn't show the CNAME). It looks like this ( show the CNAME in the long-term data / query log ) will NOT be possible, because that data isn’t stored in the database, adding this additional information would have a severe impact on performance.
The info in the query log menu does provide sufficient information.

13

Isn't the CNAME the (blocked a-0003.a-msedge.net)?

14

Yes, my point exactly. A vs AAAA is what this thread shows as in search, posts about a different subject are effectively lost.

Edit: But DL is driving now and he can move posts or not move posts as he wishes.

15

There is a difference between looking at queries in the query log menu (shows the CNAME) and looking at queries in the long-term data / guery log menu (doesn’t show the CNAME). It looks like this (show the CNAME in the long-term data / query log) will NOT be possible, because that data isn't stored in the database, adding this additional information would have a severe impact on performance.
The info in the query log menu does provide sufficient information.

16

As a personal favor, instead of editing your posts to change the content to be something that is very different from your original text, please just edit the post with additional data or strikethroughs. I'm trying to reply and you change the contents of the posts, that gets really frustrating to deal with.

18

Correct, just learned how to do that. I was just trying to get consistency in the replies, after DL6ER spilt up the topic, making it harder to understand for new readers.
Meanwhile, DL6ER has received the data, I mentioned earlier, and is looking into it.

1 Like
19

Okay, the fault is not on Pi-hole's side. Everything is working correctly. However, what @jpgpi250 sees is still real. Explanation following below.

FTL debug output for: Permitted query to ctldl.windowsupdate.com 
**** new UDP query[A] "ctldl.windowsupdate.com" from 192.168.2.227 (ID 5035, FTL 4530, src/dnsmasq/forward.c:1571)
ctldl.windowsupdate.com is known as not to be blocked
**** forwarded ctldl.windowsupdate.com to fdaa:bbcc:ddee:2::5552 (ID 5035, src/dnsmasq/forward.c:566)
ctldl.windowsupdate.com is known as not to be blocked
CNAME ctldl.windowsupdate.com
**** got reply ctldl.windowsupdate.com is (CNAME) (ID 5035, src/dnsmasq/cache.c:487)
audownload.windowsupdate.nsatc.net is known as not to be blocked
CNAME audownload.windowsupdate.nsatc.net
**** got reply audownload.windowsupdate.nsatc.net is (CNAME) (ID 5035, src/dnsmasq/cache.c:487)
au.download.windowsupdate.com.edgesuite.net is known as not to be blocked
CNAME au.download.windowsupdate.com.edgesuite.net
**** got reply au.download.windowsupdate.com.edgesuite.net is (CNAME) (ID 5035, src/dnsmasq/cache.c:487)
a767.dscg3.akamai.net is known as not to be blocked
CNAME a767.dscg3.akamai.net
**** got reply a767.dscg3.akamai.net is 2.17.107.18 (ID 5035, src/dnsmasq/cache.c:487)
a767.dscg3.akamai.net is known as not to be blocked
CNAME a767.dscg3.akamai.net
**** got reply a767.dscg3.akamai.net is 2.17.107.33 (ID 5035, src/dnsmasq/cache.c:487)
FTL debug output for: CNAME blocked query to ctldl.windowsupdate.com 
**** new UDP query[A] "ctldl.windowsupdate.com" from 192.168.2.179 (ID 6979, FTL 6074, src/dnsmasq/forward.c:1571)
ctldl.windowsupdate.com is known as not to be blocked
**** forwarded ctldl.windowsupdate.com to fdaa:bbcc:ddee:2::5552 (ID 6979, src/dnsmasq/forward.c:566)
ctldl.windowsupdate.com is known as not to be blocked
CNAME ctldl.windowsupdate.com
**** got reply ctldl.windowsupdate.com is (CNAME) (ID 6979, src/dnsmasq/cache.c:487)
audownload.windowsupdate.nsatc.net is known as not to be blocked
CNAME audownload.windowsupdate.nsatc.net
**** got reply audownload.windowsupdate.nsatc.net is (CNAME) (ID 6979, src/dnsmasq/cache.c:487)
wu.azureedge.net is known as not to be blocked
CNAME wu.azureedge.net
**** got reply wu.azureedge.net is (CNAME) (ID 6979, src/dnsmasq/cache.c:487)
wu.ec.azureedge.net is known as not to be blocked
CNAME wu.ec.azureedge.net
**** got reply wu.ec.azureedge.net is (CNAME) (ID 6979, src/dnsmasq/cache.c:487)
wu.wpc.apr-52dd2.edgecastdns.net is known as not to be blocked
CNAME wu.wpc.apr-52dd2.edgecastdns.net
**** got reply wu.wpc.apr-52dd2.edgecastdns.net is (CNAME) (ID 6979, src/dnsmasq/cache.c:487)
hlb.apr-52dd2-0.edgecastdns.net is known as not to be blocked
CNAME hlb.apr-52dd2-0.edgecastdns.net
**** got reply hlb.apr-52dd2-0.edgecastdns.net is (CNAME) (ID 6979, src/dnsmasq/cache.c:487)
cs11.wpc.v0cdn.net is known as gravity blocked
CNAME cs11.wpc.v0cdn.net

The issue here is a mis-/ or at least very strange configuration inside the CNAME path that is not very obvious. However, FTL provides all the necessary details to track what is really happening here. Let me post-process this for you in the observed case:

Let's follow the CNAME path which is really ugly for this domain:

ctldl.windowsupdate.com -> audownload.windowsupdate.nsatc.net -> ...

So far, so good. However, it turns out that there are now multiple possible branches from here on.

First branch

audownload.windowsupdate.nsatc.net -> au.download.windowsupdate.com.edgesuite.net -> a767.dscg3.akamai.net -> [2.17.107.18, 2.17.107.33]

The entire query is inspected and permitted as nothing worth blocking has been found :white_check_mark:

Second branch

audownload.windowsupdate.nsatc.net -> wu.azureedge.net -> wu.wpc.apr-52dd2.edgecastdns.net -> hlb.apr-52dd2-0.edgecastdns.net -> cs11.wpc.v0cdn.net -> [./.]

The last domain is gravity blocked and hence the entire query is short-circuited :white_check_mark:

I verified this by running various requests for audownload.windowsupdate.nsatc.net directly to Google's DNS servers. I obtained even more than the two possibilities observed above, e.g., there are also paths going through auto.au.download.windowsupdate.com.c.footprint.net, or au.download.windowsupdate.com.hwcdn.net.

You can check this for yourself by running

dig audownload.windowsupdate.nsatc.net @8.8.8.8

several times and observe that the response will be different from time to time.

TL;DR No Pi-hole error. The observe issue is caused by various CNAME paths existing in parallel. One of which contains a domain that is, by chance, on @jpgpi250's gravity list.

1 Like
21

correct, cs11.wpc.v0cdn.net is in a blocklist.

 pihole -q cs11.wpc.v0cdn.net
 Match found in https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt:
   cs11.wpc.v0cdn.net

I assume the simplest solution would be to simply whitelist ctldl.windowsupdate.com, thus eliminating CNAME checking and the misleading results?

I'm sorry to be such a pain, is the misleading result for www.msn.com, also caused by various CNAME paths existing in parallel?

22

No, we had this already before.

FTL debug output for A www.msn.com 
**** new UDP query[A] "www.msn.com" from 192.168.2.228 (ID 5006, FTL 4501, src/dnsmasq/forward.c:1571)
www.msn.com is known as not to be blocked
**** forwarded www.msn.com to fdaa:bbcc:ddee:2::5552 (ID 5006, src/dnsmasq/forward.c:566)
www.msn.com is known as not to be blocked
CNAME www.msn.com
**** got reply www.msn.com is (CNAME) (ID 5006, src/dnsmasq/cache.c:487)
www-msn-com.a-0003.a-msedge.net is known as not to be blocked
CNAME www-msn-com.a-0003.a-msedge.net
**** got reply www-msn-com.a-0003.a-msedge.net is (CNAME) (ID 5006, src/dnsmasq/cache.c:487)
a-0003.a-msedge.net is known as gravity blocked
CNAME a-0003.a-msedge.net
FTL debug output for AAAA www.msn.com 
**** new UDP query[AAAA] "www.msn.com" from 192.168.2.228 (ID 5008, FTL 4503, src/dnsmasq/forward.c:1571)
www.msn.com is known as not to be blocked
**** forwarded www.msn.com to fdaa:bbcc:ddee:2::5552 (ID 5008, src/dnsmasq/forward.c:566)
www.msn.com is known as not to be blocked
CNAME www.msn.com
**** got reply www.msn.com is (CNAME) (ID 5008, src/dnsmasq/cache.c:487)
www-msn-com.a-0003.a-msedge.net is known as not to be blocked
CNAME www-msn-com.a-0003.a-msedge.net
**** got reply www-msn-com.a-0003.a-msedge.net is (CNAME) (ID 5008, src/dnsmasq/cache.c:487)
**** got reply a-0003.a-msedge.net is (NODATA) (ID 5008, src/dnsmasq/cache.c:487)

A query

www.msn.com -> www-msn-com.a-0003.a-msedge.net -> a-0003.a-msedge.net -> [./.]

The last domain is gravity blocked. You know this already as the web interface told you (screenshot above).

AAAA query

www.msn.com -> www-msn-com.a-0003.a-msedge.net -> a-0003.a-msedge.net -> [NODATA]

The last CNAME domain is not checked as we already know that it is NODATA so no need to do the (expensive!) database checking as there is nothing that could be blocked.

23

Thank you for your time, effort and persistence, to look into a problem, that in the end turns out to be just another faulty list entry.

I assume the code to show the CNAME entry in the query log

Screenshot at 2020-01-29 22-57-40
Screenshot at 2020-01-29 22-57-40935×108 11.2 KB

will sooner or later appear in the beta5 branch, so everybody can enjoy this great addition.

Thanks again, anything else you want me to test, just ask...

1 Like
25

I noticed that.
You may have the following example for fun / eehhm... testing:

  1. add trafficmanager as regex to the blacklist (stands for trafficmanager.net, a tracker covered as "load-balancer").
  2. try to visit this site: https://www.mein-edenred.de/my-cards
    Here you can see exactly the behavior that seems that there's a diff'rence between A and AAAA-analysis and -blocking. Here it is easy to find out that trafficmanager was the problem.
    Now:
  3. Try to visit the following sites (with trafficmanager still in the regex list):
    shim-pri.wifiradiofrontier.com
    medion.wifiradiofrontier.com
    medion2.wifiradiofrontier.com
    These are just three domains (!) that a Medion Network Radio will connect to play Internet Radio directly. It took hours to find out that trafficmanager was blocking everything in these cases. I was short before using wireshark to analyze everything that's going through the router. But via traceroutes, pings and trying to browse all the CNAMEs behind these domains step by step trafficmanager turned to be a big and very good hidden problem in CNAME inspection.
26

With the new code (there are several screenshots floating around in here, one in the post above), the dashboard will exactly show you which domain in the chain was the reason for blocking.

27

Yes. I tried to checkout already, but it won't work. Don't know why. Tried again just a few moments ago and now i can see the blocked domain in a CNAME. Great!
mny tnx 73

28

It's always the same: Browser cache. There isn't all that much we can do about it, but things settle down after a little bit of time.