Implement wildcard for blacklist (any subdomains of a domain)

Feature Requests Implemented
1

This post is following the discussion in the HELP section Subdomains not being blocked V3.1.4

Request
Today (V3.1), wildcard blacklisting can be performed by adding an entry in the dnsmasq configuration files (or via the UI). Yet, this is not scalable and, for porn for instance, it would be great to be able to blacklist all the subdomains of a given domain (eg www.porn.com, cdn1.porn.com, cdn2.porn.com, etc.). There are blacklists with over several hundreds (even millions) of entries. We cannot key them in dnsmasq configuration files for the performance shall drop.

Expected Behaviour:
Using lists that contains domain.tld should get pi-hole block both “domain.tld” and any “anything.domain.tld” subdomains?

There could be two kinds of lists :

  • First without default wildcard implementation
  • the other with wildcard implementation

This would permit to implement or not the wildcard depending on the list (example : adds ==> no need ; porn ==> needed)

Actual Behaviour:
Only the “domain.tld” is blocked. Pi-hole is not enforcing the wildcard blocking with lists.

Here are logs that shows it (50.131 is the client - 50.1 is the router+default gateway - 50.55 is the Pi-hole)
Sep 8 23:29:25 dnsmasq[4743]: query[A] myfreecams.com from 192.168.50.131
Sep 8 23:29:25 dnsmasq[4743]: /etc/pihole/gravity.list myfreecams.com is 192.168.50.55
Sep 8 23:29:25 dnsmasq[4743]: query[A] myfreecams.com from 192.168.50.131
Sep 8 23:29:25 dnsmasq[4743]: /etc/pihole/gravity.list myfreecams.com is 192.168.50.55
Sep 8 23:29:25 dnsmasq[4743]: query[AAAA] myfreecams.com from 192.168.50.131
Sep 8 23:29:25 dnsmasq[4743]: forwarded myfreecams.com to 192.168.50.1
Sep 8 23:29:25 dnsmasq[4743]: query[AAAA] myfreecams.com from 192.168.50.131
Sep 8 23:29:25 dnsmasq[4743]: forwarded myfreecams.com to 192.168.50.1
Sep 8 23:29:25 dnsmasq[4743]: reply myfreecams.com is NODATA-IPv6
Sep 8 23:29:30 dnsmasq[4743]: query[A] www.myfreecams.com1 from 192.168.50.131
Sep 8 23:29:30 dnsmasq[4743]: forwarded www.myfreecams.com1 to 192.168.50.1
Sep 8 23:29:30 dnsmasq[4743]: reply www.myfreecams.com1 is 207.229.73.118
Sep 8 23:29:30 dnsmasq[4743]: reply www.myfreecams.com1 is 207.229.73.117
Sep 8 23:29:30 dnsmasq[4743]: query[A] www.myfreecams.com1 from 192.168.50.131
Sep 8 23:29:30 dnsmasq[4743]: cached www.myfreecams.com1 is 207.229.73.117
Sep 8 23:29:30 dnsmasq[4743]: cached www.myfreecams.com1 is 207.229.73.118

Same with a remote client made another with yes24.com
C:\Windows\System32>nslookup yes24.com
Serveur : raspberry
Address: 192.168.50.55

Nom : yes24.com
Address: 192.168.50.55 <== expected and the browser shows the “blockpage”

C:\Windows\System32>nslookup www.yes24.com
Serveur : raspberry
Address: 192.168.50.55

Réponse ne faisant pas autorité :
Nom : www.yes24.com
Address: 61.111.13.51 <== unexpected !

2

For the cases where it doesn't block the www. version, that could be the list maintainer's fault. Alternatively, we could add a blacklist entry for the www. version of every blacklisted domain.

3

The this is this does not only concern www but many things. If you look at porn / other maintained for squidguard you'll see they only put the domain and not all the potential subdomains. The reason ? This would multiply the size of database and the subdomains keep on changing.

Hence the FR.

4

I wish I could vote for this but the website won't allow me.

I would also like see this for several reasons

  1. Some lists are meant for wildcard blocking like yoyo - it is evident by looking at the contents of the list it self, and it's also mentioned in their web page.
  2. Adding huge number of address entries for wildcard does not change dnsmasq's loading time or memory the least bit (same as host entries). As a comparison, I have noticed a slowdown in dnsmasq startup when loaded with a big list of cname entries.
  3. Domains and lists are the core of pi hole :upside_down_face:
5

There have been performance issues with dnsmasq and loading domains via config files in the past: Improving The Pi-hole's Performance For Blocking Ads

6

Increased speed by using /etc/hosts instead of a dnsmasq config file
https://jacobsalmela.com/2015/06/06/improving-the-pi-holes-performance-for-blocking-ads/

As you rightly mentioned, that was in the past. It has been two years since, and I have noticed no such issues with dnsmasq v2.76. The option for making a list as wilcard/address entry is, IMHO, still a valid feature request.

7

Regex blocking in v4.0 should implement this.

https://docs.pi-hole.net/ftldns/regex/overview/