Apply Pi-Hole blocking to CNAMEs

What does

sqlite3 /etc/pihole/gravity.db "SELECT * FROM vw_gravity WHERE domain = 'liberation.eulerian.net';"

return?

Just wanted to give a big thanks for the team for even looking at this! THANK YOU!

@DL6ER has been banging it out folks.
I know you have been knee deep in it. Thanks again!

Commits on Nov 21, 2019

1. [Add pure attribute to vGet() to signal that this function might be el…](https://github.com/pi-hole/FTL/commit/7fd801dbc250fd706bdf408a624619fee96b75f4)

[ ![@DL6ER|20x20](https://avatars3.githubusercontent.com/u/16748619?s=60&v=4) ](https://github.com/DL6ER)

[DL6ER](https://github.com/pi-hole/FTL/commits?author=DL6ER) committed 2 days ago

[Add a generic dynamic vector implementation. I tested my implementati…](https://github.com/pi-hole/FTL/commit/0baa56c3c3a7486ff48515a050f8df8ec45645db)

Commits on Nov 22, 2019 

[ ![@DL6ER|20x20](https://avatars3.githubusercontent.com/u/16748619?s=60&v=4) ](https://github.com/DL6ER)

[DL6ER](https://github.com/pi-hole/FTL/commits?author=DL6ER) committed 2 days ago

[Fix: Only set domain as not-to-be-blocked when this is true.](https://github.com/pi-hole/FTL/commit/4269897b58bbacca732e6275198a8c0ceecaf803)

[ ![@DL6ER|20x20](https://avatars3.githubusercontent.com/u/16748619?s=60&v=4) ](https://github.com/DL6ER)

[DL6ER](https://github.com/pi-hole/FTL/commits?author=DL6ER) committed yesterday

Signed-off-by: DL6ER <dl6er@dl6er.de>

[Add more debugging output.](https://github.com/pi-hole/FTL/commit/7fc340c16affbc70de72146792f4310bf12bf5e0)

[ ![@DL6ER|20x20](https://avatars3.githubusercontent.com/u/16748619?s=60&v=4) ](https://github.com/DL6ER)

[DL6ER](https://github.com/pi-hole/FTL/commits?author=DL6ER) committed yesterday

[Remove static instruction from query_blocked() as it is called from p…](https://github.com/pi-hole/FTL/commit/25a403a7c4ddb45e70c062795e602a1b8296fa5b)

[ ![@DL6ER|20x20](https://avatars3.githubusercontent.com/u/16748619?s=60&v=4) ](https://github.com/DL6ER)

[DL6ER](https://github.com/pi-hole/FTL/commits?author=DL6ER) committed yesterday

[Improve performance significantly. We store if (and if: why) a certai…](https://github.com/pi-hole/FTL/commit/faac7005d173dd95bf654addfedbcfd39ee0a421)

[ ![@DL6ER|20x20](https://avatars3.githubusercontent.com/u/16748619?s=60&v=4) ](https://github.com/DL6ER)

[DL6ER](https://github.com/pi-hole/FTL/commits?author=DL6ER) committed yesterday

You're assumptions are correct. If it isn't "BAD", it shouldn't be in a blocklist. It's always possible to whitelist it.

Could not agree more.
Also, is that toggle already in uBlock Origin? Can't seem to find it. Thanks

I implemented CNAME during-the-path blocking for branch new/internal-blocking. This is the branch allowing also per-client blocking rules.

There are two points worth mentioning:

  1. Blocked replies are currently replied to with NODATA instead of the configured blocking mode (technical limitation, which does not yet hurt at this point).
  2. More tests are outstanding.

One of the tests I did:

  • Removed f7ds.liberation.fr from gravity
  • Added atc.eulerian.net to my exact blacklist

Result:

$ dig f7ds.liberation.fr

; <<>> DiG 9.10.3-P4-Debian <<>> f7ds.liberation.fr
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42224
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;f7ds.liberation.fr.            IN      A

;; Query time: 19 msec
;; SERVER: ::1#53(::1)
;; WHEN: Mon Nov 25 12:59:54 CET 2019
;; MSG SIZE  rcvd: 47

A post was split to a new topic: Issue with Gravity (development branch)

Thank you guys for working on this topic so hard!

In the meantime, it seems there is another tracking company actively suggesting the CNAME method:

Measure accurately with first-party tracking

Gain at least 20% more data compared to other players in the market. With Ingenious, you bypass ad-blocking technology and collect accurate data for accurate analysis.

According to this Github comment: Address 1st-party tracker blocking · Issue #780 · uBlockOrigin/uBlock-issues · GitHub

Looks like they recommend their customers to use marketing.net.*.

Example:
Non-authoritative answer:
marketing.net.brillen.de anonical name = tr-brillen-de.affex.org.
tr-brillen-de.affex.org canonical name = lb1.affex.org.
Name: lb1.affex.org
Address: 35.187.117.15

Non-authoritative answer:
marketing.net.home24.de canonical name = tr-home24-de.affex.org.
tr-home24-de.affex.org canonical name = lb1.affex.org.
Name: lb1.affex.org
Address: 35.187.117.15

I hope it might help you to get them blocked as well.

If

is on gravity or your personal block list, it should(TM) already work.

If you block all domains containing or starting with "marketing.net" using regex, this fixes this specific problem.

I'm really trying to follow the analogies but I can't relate this to DNS in any way.

1 Like

Friendly reminder to all participants in this thread. Please lets keep the discussion to specifically discussing this feature request in a technical manner. No sales pitches need to be made, we're already working on it.

No need for analogies or hyperbole. I'd really rather not have to lock the continued discussion of this topic down to developers only, but if we get much more noise in the thread then it will be difficult for newer readers to follow along with.

This has been a friendly mod announcement. Lets keep things on topic. Thanks! :heart:

4 Likes

I have now new/internal-blocking active on my backup Pi-hole so I don’t have to revert every time.

Removed f7ds.liberation.fr from gravity:

sqlite3 /etc/pihole/gravity.db "DELETE FROM gravity WHERE domain='f7ds.liberation.fr';"

atc.eulerian.net is added to exact blacklist.

pihole restartdns to be sure and then ran dig.

I have a NOERROR instead of the expected NODATA and the query time is 12ms. For local resolving is less than 2 ms and 12ms is also not a external resolve which is above 40ms.

; <<>> DiG 9.11.5-P4-5.1-Raspbian <<>> f7ds.liberation.fr
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35367
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;f7ds.liberation.fr.            IN      A

;; Query time: 12 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Nov 27 16:47:03 CET 2019
;; MSG SIZE  rcvd: 47

Taking atc.eulerian.net from the exact list then it is not blocked anymore:

; <<>> DiG 9.11.5-P4-5.1-Raspbian <<>> f7ds.liberation.fr
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10563
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;f7ds.liberation.fr.            IN      A

;; ANSWER SECTION:
f7ds.liberation.fr.     1710    IN      CNAME   liberation.eulerian.net.
liberation.eulerian.net. 5310   IN      CNAME   atc.eulerian.net.
atc.eulerian.net.       5310    IN      A       109.232.197.179

;; Query time: 10 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Nov 27 16:50:19 CET 2019
;; MSG SIZE  rcvd: 118

Those 12 ms is not a problem and the CNAME is blocked and so success!

In my earlier version of this posting I thought that new/internal-blocking was merged into FTL development but it seems to be the other way around.

The delay (likely) comes from that we don't inject the CNAME into the DNS cache when the path is blocked half-way through. Hence, the CNAME path needs to be re-preocessed every time. I might be able to come up with an improved implementation but all things DNS cache are rather tricky to get right...

@anon55913113 Can you check in your pihole.log if this query gets forwarded at all or if the 12msec are truely local processing only? If so, how many domains do you have in gravity, and the black- and whitelists?

No, what you see is what is commonly called NODATA. This the response code NOERROR plus an empty response for the requested record (A in your example). The only other negative reply could be NXDOMAIN. This would show up with response code NXDOMAIN and no response for the requested record.
There is no response code NODATA on its own.

A post was split to a new topic: Gravity database issue (branch new/internal-blocking)

The per-client blocking feature includes intermediate CNAME path blocking.
We have been going back and forth on this and came to the conclusion that it is worth delaying the arrival of Pi-hole v5.0 yet more to include both features already here.

Stay tuned!

3 Likes

5 posts were split to a new topic: Automated CNAME mapping to an IP

A post was merged into an existing topic: Automated CNAME mapping to an IP

Implemented with v5.0

Closing to release votes.