Implement Response Zone Policies (NXDOMAIN) for end-user performance increase

Of course, but I would only find that useful if I actually needed a blocking page. Now we're forced to have blocking page traffic to the Pi because of the design. I would make the blocking page feature optional and NXDOMAIN default. Basically like editing your own HOSTS file to point 127.0.0.1 instead of some rando site (or your router) to get 404's from it.

I have a test environment set up and running that does what you suggested, however, it has (at least) two significant drawbacks:

  1. It is not possible with the version of dnsmasq shipped with the majority of the systems (including but not limited to current Raspbian) - v2.73. However, if you compile dnsmasq from source (2.76-37-g5eb9dde) you could do it.
  2. As @Mcat12 already mentioned: the blocking page is not display anymore

Since we have to take care that everything is working nicely (i.e. no compiling of source code on the local machines), I consider this as out of scope (at least for the moment).

Might moving to "bind" be an option for this?

We have experimented a long time ago with redirecting to 0.0.0.0, but that resulted in a lot of frowny faces filling up sites where the ads used to be. Also, that would result in no blocking page. We might think about having it as an option in the future. Also, redirecting to something like local host will result in lots of time outs and make the page appear to still be loading when it's actually just waiting on time outs.

Responding with NXDOMAIN is not the same as responding with 0.0.0.0 or 127.0.0.1. In the last case it will result in more delay rather than less. An NXDOMAIN will directly inform the client that there is no content coming from the requested host, so any future requests will be omitted, resulting in a speed-up. It may indeed result in frowny faces here and there, but the speed increase is worth it.

I have implemented such an approach using bind (so without Pi-Hole) and it works great. No frowny faces (but I guess that's a Chrome thing), just empty space where ads would've been.

Have you measured the speed increase? There is a speed increase with just normal Pi-hole anyways, since it's getting a very small response from a local server.

If you have the technical capability to set up bind then you are already way above where most of our users feel comfortable. I agree that NXDOMAIN is a faster way to handle things, but looking at the situation from our biggest user base, the home user that is running the install script on a Raspberry Pi, that level of sophistication just isn't needed. The biggest thread of the last few weeks on our Reddit sub has been how to deal with family members getting angry that their favorite sites are blocked. They need the ability to whitelist via webpage and to let their users know that sites are blocked because of Pi-hole. If they just returned a blank insert or no page at all the number of support requests for "The Pi-hole broke my internet" would skyrocket.

We have looked, and are again looking at how to modularize the code so that you can put in your own resolver, but with our dependency on dnsmasq for DHCP service, that will pretty much likely always be the default in an install, and the option would be for more sophisticated admins to adjust the settings to their comfort. There's only so much you can do with shell scripting and bash, and were already over 1100 lines of code just for the basic-installer.

I do appreciate the request, and this is the best place for the feature to be voted on and moved to a higher level of priority if needed, but with our extremely limited resources, we can only do so much so fast.

@DanSchaper I have to agree that if it happens frequently that "important" websites are blocked by Pi-Hole, it's better to return an informative message stating why it's blocked rather than just not getting any response.

However, I'm wondering: doesn't that mean that the default block list is too restrictive? In the Bind-configuration I mentioned above, I'm using the blocklist from MVPS. It blocks nearly all advertisements and trackers, but I never ever run into problems where sites do not work.

Anyway, I really appreciate the Pi-hole project even though I'm currently not using it. Lack of this feature is one of the reasons. It may be something you can add as an additional setting that is turned off by default, so that users can explicitly and deliberately turn it on when they desire the extra performance improvement it brings.

Can you specify what realistic performance improvement this would bring? Pi-hole itself improves web page performance by returning a very small placeholder instead of ads. I don't think this method would make such a performance boost that users would notice.

The frequency of seeing the Pi-hole block page is entirely up to the users browsing habits. Most of the devs run the default lists, and I for one have never seen the block list show up in the wild. Some users may use Google's shopping integration with search, which might cause them to be redirected through domains which are blocked, so that is when they'd whitelist that domain. Many users actually ADD lists, because they feel they need more domains blocked.

That assumes that the default block lists are the only lists enforced. We have users that often add in highly restrictive block lists and break things, with only the block page to let them know that they may be a little too aggressive in their actions. We have to balance the power users with the first timers and get a product that is fair and useful to everyone.

Being far on the power user side myself I would like to see us doing a lot more, but again we have very limited resources and far fewer users on the power end of the scale. This is a situation we are looking at how to best resolve, but swapping out the entire back end of the Pi-hole isn't something that is quite easily accomplished, while keeping the new user friendliness intact.

1 Like

My apologies for digging up this quite old feature request (it's not closed yet !), but actually I find it interesting and upvoted it.
I totally understand that the "blocking page" feature is useful for most of users (even if with HTTPS almost everywhere it's becoming less and less useful) and PiHole should keep that feature by default.
But that would be useful to have the ability to configure PiHole so it sends NXDOMAIN for blocked queries, not much for performance increase (which would certainly be almost negligible as stated earlier) but to avoid having to set IPTables rules on PiHole to block TCP / UDP port 443 traffic (otherwise some websites got slow).
The UDP rule don't bother me, but the TCP rule prevents HTTPS hosting (or makes it unconvenient for the least).
OK, that concerns only a few people (me !), but even if you forget about HTTPS hosting, you have to admit it would be "cleaner" to be able to send NXDOMAIN (or even a custom IP for those wanting to have their own blockpage ?) instead of replying with PiHole IP, then generating a HTTPS connexion on the client side that will be blocked by Pi's IPTables.
Don't know what amount of work is required, but if such a feature was added (without modifying PiHole's default behavior), it could be useful to a few users (but I totally understand "a few" is no priority !)

Here is a relevant pull request

1 Like

Sorry, I didn't see this one, thanks for the link !

1 Like

An updated link.

1 Like

I've just tried the "BLOCKINGMODE=NXDOMAIN" setting and so far everything seems to be doing fine : domains from blocklists are still blocked, and DNS replies sent by PiHole for those domains are "NXDOMAIN" as expected, very nice, thanks !

2 Likes

I saw this feature request and feel that it is worth my adding a comment here rather than a brand new one.

I have recently become interested in pi-hole - or something like it - as a possible delivery mechanism for RPZ in the SOHO user space (disclosure I work for a company - ThreatSTOP - that does DNS Firewalling and offers a commercial DNS service blocking malware ).

So let me start with a quick description of RPZ as it is implemented by Bind and other DNS servers. RPZ is a way to change the DNS response to a request for a domain that hits certain criteria. One of the responses can be NXDOMAIN (i.e. no answer) other options include an A record (which is more or less what pi-hole does), a CNAME redirect (redirect bad.domain to warning.page) and some others including (PASSTHRU - i.e. no action). RPZ is also a mechanism for transferring block/redirect data to remote DNS servers using zone transfers with an automatic update timer (the zone TTL).

Note that RPZ can have rules that act not just on the domain name, but also the IP address it resolves to (and on the domain's nameserver names/ip addresses).

It seems to me that pihole could usefully benefit from some of the features of RPZ not just the NXDOMAIN vs redirect thing.

Firstly a number of organizations make block data available as an RPZ. Some of these are free, some (e.g. ours) are paid products. It would be worth writing code to allow those sources to be imported into pihole (the code for retrieving and importing domain data is literally a few lines of shell/sed script plus the dig command).

Secondly it would be useful to have the "act on IP address" capability as well as domain capability. Since you are hacking dnsmasq you might want to consider adding this?

Thirdly and relatedly while the load of serving a default "blocked" webpage instead of nothing is not that great for a single user, at a larger, enterprise scale, it can be - especially if the page is being served to a bot that is trying unsuccessfully to call home.

As far as I recall, Domain Name Service Response Policy Zones (DNS RPZ) are just a method allowing us to provide alternate responses to queries. It's been officially called like this in BIND9.8+ and sometimes even referred to as "DNS firewall" (which I find a bit confusing, but tastes differ).

Being based on dnsmasq as our internal DNS resolver, we did not attack this feature request by implementing a new zone handling method. Instead, we use the existing machinery that loads HOSTS entries into the resolvers cache. For making them persistent, we set their TTL value to infinity as we don't expect them to expire while the DNS server is running.

If a user decides to chose the legacy way of blocking, we register for each domain A and AAAA records that will be served on any query of this domain. If a user instead set to use NXDOMAIN blocking, then we populate the cache not with A or AAAA records, but with negative records, such that dnsmasq can immediately reply to corresponding requests with NXDOMAIN (as if it would have tested it before and got told that the record does not exist).

I don't really get what you suggest to serve instead of A and AAAA or NXDOMAIN blocking? Right now, I don't see any real benefit a CNAME would bring.

I'm not sure how this could be any different to just a list of blocked domains. Can you provide an example for a free list?

This may or not be useful, what you you expect such an action could be? Running a script or sending out an alarm email? However, a new feature request seems mandatory for such a thing as many details seem to need clarification.

I don't really understand your suggestion here. Do you mean serve the blocking lists using A and AAAA such that (human) users see a blocking page while having a second list that blocks additional domains for which NXDOMAIN would be served?

1 Like

Great, this is what I want to use. How can I change my FTLDNS install to this one?

Forgot to quote...

You can find instructions here:

https://docs.pi-hole.net/ftldns/configfile/#blockingmode