Option to block recently created domains (DGA)

Pi-hole is mainly for blocking ads. But it also can serve to stop malware.

A fairly simple and effective way would be to use Creation Date of a domain from a simple whois lookup. If the domain was registered less than a specified period of time ago, it would be pi-holed.
I think it happens rarely that you visit a newly created domain, but it should be opt in anyway.

whois pi-hole.net
   Domain Name: PI-HOLE.NET
..
   Creation Date: 2015-03-20T18:00:23Z
...

Related:

I think this is outside the scope of Pi-Hole; it is not intended to be a one-stop internet security program. It is a DNS resolver.

If you foresee potential interest in an enhancement that would do this, write a script that would find domains recently queried in the query log, perform the whois search you mention, and then add to blacklist as desired. Put the script on GitHub for others to use.

I agree that this request is an edge case for the scope of pi-hole.

You do not want to slow down resolving DNS requests by a whois lookup. This has to happen afterwards. But adding it to the blacklist permanently is not something I suggest to do.
As an example: Domains that are less than a week old should be pi-holed. After that period of time the domain should be accessible.

A script that will blacklist domains can not serve this purpose. Maybe that solution can be considered when we get an option to blacklist domains for a certain period of time and not permanently.

Theoretically the script can be made to remember and remove those domains from the blacklist after a period of time. This would make the script more complicated.

I already have seen scripts that blacklist domains and remove them after a period of time for the purpose of parental control.

Why not? The same script that can put a domain on a blacklist based on the age of the domain registration can easily remove it from the blacklist based on the same criteria. Run the script once a day and check everything in the blacklist, or have a separate table that the script creates and when the time is up on an item, remove it from the blacklist the next time the script runs.

It should be more responsive than once a day, more like right after a non cached request. And that criteria will apply to domains that were blocked manually.

The only way to do it with a script would be to save the blacklisted domains and the time they need to be unblocked again. Basically implementing blacklisting for a period of time that IMHO should also be a feature.

I am not saying solving it with a script is not possible. Just doing it within pi-hole will be more efficient.

Anyway this probably has to wait as other features are more pressing. I just wanted to throw in the idea.

I support your request slawa.
Funny DNS names are getting a serious problem, not just for malicious but also for add sites.
The last month I checked a few new ones, they were not blocked by using upstream OpenDNS or Quad9.
Upstream filtering is only done if half the world starts complaining :slight_smile: .
So yes, I believe it's a great optional feature for PiHole.

1 Like

Another example where this feature will help and is in scope of Pi-Hole.

There are some websites that use aggressive ads tactics.
They serve their ads from a new domain they create every few days.

I block the ad domain and few days later the ads pop up again.
I can effectively block it on Desktop with addons.
But some mobile phones can not use uBlock.

Here are the sites:
[WARNING!] Ads are NSFW

Ads are NSFW

And why would regex blocking not work in these cases?

It works for a few days. Ads come from new domains after a while. And the circle begins.

I have a dozen domains blocked that were used for ads. And new domains keep coming.
Usually they use a script that tries 3-4 domains to display ads.

Not sure this will help,

Found this site. If somebody is capable of generating lists from this site, you could add them to your blocklists, on a daily base.

There a so many new domains created. This would be overkill.
And then it should not block them permanently, just for specified amount of time.

Here are some specifics what I am thinking about:

  • There should be a cache of already checked domains. This can use the cache the pi-hole already has and add a flag [creation_date_checked:true/false]. This will save whois calls on frequently used domains.

  • Add a new table to the local database that handles a temporary block list. This can be a useful feature for the whole project. e.g. you can block Facebook for a week without the need to manually unblock.

  • Add a function that will run separately after domain resolution:

  1. Check if domain in cache and has true flag
  2. Check WHOIS Creation Date on cache miss
  3. If new domain, add to temporary block list with the desired duration (Creation Date + new domain block length from the settings)
  4. Domain will be automatically unblocked after temporary block expiry

In the Setting you should be able to specify for how long domains will be blocked after their registration.

This will help block DGA domains. Ad domains can circumvent it buy buying random old domains for cheap, instead of registering new ones.

This sounds more like a request for a new list than a feature for Pi-hole.

You could write a script that does this by itself. In the current development version of Pi-hole (and the next v5.0 release), we store the creation date of blacklist values.

You may also want to read this discussion:

Also in the current development version, we now support massive whitelists. The idea of above's discussion is that they want to permit the, say, top 10 million domains with sufficient "reputation" (however you measure that) and block everything else.

It's naive to think WHOIS will always output "Creation Date". Try running WHOIS on yandex.ru for example and it says "created" instead of "Creation Date". There is no standard WHOIS output format. I could not find any way to specify the creation date as an attribute but it might be possible.

The issue with newly-created domains is getting more attention:
https://unit42.paloaltonetworks.com/newly-registered-domains-malicious-abuse-by-bad-actors/

1 Like

Nextdns.io has introduced a very useful function to fight Phishing and Malware: the blocking of domains registered for less than 30 days.

New domains are often used to spread Phishing or Malware in Ad Hoc campaigns.

I think this function is very useful, I would also like to see it on Pi-Hole.

Thanks

1 Like

Pi-hole provides just the means to filter DNS traffic, not the filters themselves.

You are free to configure Pi-hole to use any blocklist of your choice, so you can implement this straight away by employing a corresponding blocklist.

With regards to the idea itself, I am skeptical whether this is a viable approach.
AFAIA, there are no binding standards guiding how a registrar should store and alter registration information, nor which parts of that information is made accessible, nor how that is done.

In addition, it leaves to consider what defines a newly registered domain, as e.g. change of ownership from a trustworthy to a malevolent site content provider does not necessarily change that site's registered on date.

1 Like

Great news then, you've always been able to do that with Pi-hole. Just add the list of the domains and they will be blocked. Starting with version 5 they will even be blocked when they appear as CNAMEs.

The point of the feature is that such a list is automatically generated from the domains the users of pi-hole visit and domains that are out of age threshold are automatically removed from being blocked.

Suggestion to 'manually' create such a list has issues:

  • creating a list of all newly registered domains worldwide is difficult (cooperation from registrars is required?)
  • such a list including all new domains would be large and overkill for this task (11+ million new domains every 90 days some only active for a few hours)
  • there would be one point of failure if the host of such list goes down since that list needs to be constantly updated (like every hour) to be effective
  • pi-hole updates lists automatically only once a week!
  • a list can only have domains with one age threshold (non variable)

I see the problem that WHOIS responses do not always include creation date or name it differently. Over long term registrars would be inclined to provide it to prevent abuse.

If we had to do a WHOIS check and evaluation for every client DNS name query then latency would be on the order of seconds per query.

I mentioned before that for performance this should run separately after DNS request. But it will result in the very first domain request not being blocked. This is not ideal as malware might only request the DNS resolution from cnc servers once, but for other use cases it is enough.

I try to elaborate on my previous post:

The whole WHOIS protocol lacks proper standardisation, e.g. it does not guide encoding at all -neither for queries nor for the requested content- which would be a prerequisite for internationalisation, many of the data are not required to be well-formed (e.g. data might be labeled differently by different providers or not have the same semantics tied to it), there is no standard for determining the authoritative WHOIS server for a given domain name, possibly leaving you with no or incomplete information, etc.
(Just do a whois on google.com and heise.de to get an idea of what I am talking about, and mind that output would also differ by the whois binary in use.)

Worse still, to my knowledge, access to the data you wish to scrutinize here is currently not standardized at all, nor are registrars obliged to offer programmatical access to that data for free (afaik, only mailing and email address as well as phone number are required to be publically available - but subject to be adopted to latest GDPR requirements).

They also commonly enforce rate limits on the amount of queries they would allow from an IP address, or may offer to lift such limits for a fee.

As accessing and correctly extracting or interpreting that data is tricky, numerous service providers have established a business around this by offering access to well-formed proprietary APIs.
So rather than wasting resources on redeveloping something that is already available, Pi-hole offering integration with one of those APIs could be an option.

However, employing such a service would require you to enter a contract with the provider of your choice (e.g. 50 bucks for requesting max 24k domains - per month, that is).
And then there may be additional contractual obligations, e.g. no permission to store requested data locally, which would restrict you from caching data over a prolongued period, that in return would have to be honored by Pi-hole in order to not make you transgress those obligations.

All of this to block you from accessing the website your friend has just started when opening her new cafe?

Why just block new domains? And why suddenly allow them after some 30 days? I doubt that 'new' domains are any more or less trustworthy than existing ones (avoiding having to define the exact meaning of 'new') or -if they are malevolent- that they would cease being so after an arbitrary period of time.

When operating at that level of confidence or mistrust, maybe employing a deny all strategy would be worth considering, and certainly easier to implement.

1 Like