Best way to "programatically" handle blocking for wildcard TLDs?

Help
1

Hi all,

I'm brand new to the Pi-hole scene, so apologies if this is a dumb set of questions!

I'm looking to block requests to a good amount of the TLDs out there (see http://data.iana.org/TLD/tlds-alpha-by-domain.txt for the authoritative list). I wrote a script to parse out the top 100-200 that I'd still like to query, and so I'm left with about ~1300 TLDs to block all wildcard/subdomain requests for.

A few questions here:

  1. Can this be done with a block list? It seemed that block lists (e.g., the default ones) call out the exact domain in question to block (and do not allow for a wild card), but my understanding about blocklists vs. blacklists may be very incorrect here.
  2. If this needs to be done with a script that interacts with Pi-hole - what is the best way to do that? Should I iterate over the file one line at a time, and just call pihole -wild TLD1, pihole -wild TLD2, pihole -wild TLD3, etc, which may take a while? Or does Pi-hole's ability to support multiple domains at once allow a user to specify a ridiculously large number of them? (e.g., pihole -wild TLD1 TLD2 TLD3 ... TLD1299 TLD1300)
  3. Alternatively, assuming that I need to use a script, is putting an arbitrary file in /etc/dnsmasq.d (per https://discourse.pi-hole.net/t/how-do-i-add-wildcard-sites-to-the-blacklist/337) with the TLDs formatted in the right format a better solution, or is that frowned upon?

My goal is to be able to re-apply my entire "bad" TLD list on a weekly basis, but wanted to know what the recommended path is for doing so from a developer perspective.

Thanks,
Andras

2

Yes, just add the TLDs to the wildcard blacklist and you're set up with blocking whatever is requested under these TLDs.

3

You may also be interested in:

https://pi-hole.net/2018/04/24/blocking-via-regex-now-available-in-ftldns/

4

Good to know - thank you! I was able to rebuild my Pi2 early this morning; pihole -wild was able to take 4000 domains as arguments and processed them in about 10-12 minutes.

5

Hmm, this is much (!) longer than I would have expected, but then I have never tried the wildcard feature with more than maybe 5 domains. Do you notice any weird slowdowns or is everything working fast after having added them all?

6

Haven't done much post-load testing as I mocked it up for PoC purposes, but seems to be working fine so far. I did a few dig commands to validate and they looked good.

And to expand on the time for a bit - it took about 6-7 minutes for bash to process the command itself, and the pihole process another 5 minutes to add all 4,000 of those arbitrary TLDs (three-letter strings aaa ... fzz just for testing purposes). So pihole seemed to take them all in rather nicely - they loaded in the GUI just fine as well (took an extra second or two, of course). I'll play with it more in the coming days to see if there are any oddities.

2 Likes
7

You don't have to do further testing. Glad to hear that everything worked nicely for you :slight_smile:

8

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.