The current way of working of Pi-Hole is that as a client, for example, doubleclick.net requests, the device its own IP address, say, 192.168.1.100, gives an answer. What does the client then subsequently in its browser is a connection set to 192.168.1.100 and there retrieve the requested resource, which will undoubtedly end in 404. Your Pi Hole thus has to deal with many HTTP requests (depending on how many clients you're behind), and your client is still doing a lot of useless requests.
It would be better to use Response Zone Policies. Your DNS server will not answer 192.168.1.100 but a NXDOMAIN instead. This means something like: This domain name is not registered and therefore will be in a "unknown host" error printouts on the client. The client does in this case not even a request to a server (because he has no IP address), and the server will handle much fewer requests. This only provides a real speed gains.
There is a difference in what you describe and what we do:
that is true, but we serve a page for all 404s showing either the blocking page or an empty page (depending on the content).
The response time with my Raspberry Pi B (Version 1) with a slow SD card and running some other stuff on it is on the order of 0.2 seconds. I think that is quite acceptable.
Although this best effort situation works, I don't understand why you would actually want that traffic to happen in the first place. It's like having a blacklist for telephone numbers to call on your switchboard, and instead of directly letting the switchboard respond in 'not a valid number' tone, it does connect to a secondary telephone service, set up the call, let the other service respond 'no I don't have what you are looking for' and then disconnect the call. A whole set of useless actions that will only slow things down without actually providing anything useful.
I don't also see this scale out for a bigger network. Say you have an apartment building with 20 households which all have a laptop, smartphone, tablet, etc. You then get all those useless 404 connections to the Pi, while you could just serve the NXDOMAIN DNS response and be done with every attempt in milliseconds instead of hundreds of milliseconds as you've shown.
The custom blocking page needs to have these web server requests so that it can be shown if a user goes to a blocked domain. We can't do that just through DNS, without the web server.
Of course, but I would only find that useful if I actually needed a blocking page. Now we're forced to have blocking page traffic to the Pi because of the design. I would make the blocking page feature optional and NXDOMAIN default. Basically like editing your own HOSTS file to point 127.0.0.1 instead of some rando site (or your router) to get 404's from it.
I have a test environment set up and running that does what you suggested, however, it has (at least) two significant drawbacks:
It is not possible with the version of dnsmasq shipped with the majority of the systems (including but not limited to current Raspbian) - v2.73. However, if you compile dnsmasq from source (2.76-37-g5eb9dde) you could do it.
As @Mcat12 already mentioned: the blocking page is not display anymore
Since we have to take care that everything is working nicely (i.e. no compiling of source code on the local machines), I consider this as out of scope (at least for the moment).
We have experimented a long time ago with redirecting to 0.0.0.0, but that resulted in a lot of frowny faces filling up sites where the ads used to be. Also, that would result in no blocking page. We might think about having it as an option in the future. Also, redirecting to something like local host will result in lots of time outs and make the page appear to still be loading when it's actually just waiting on time outs.
Responding with NXDOMAIN is not the same as responding with 0.0.0.0 or 127.0.0.1. In the last case it will result in more delay rather than less. An NXDOMAIN will directly inform the client that there is no content coming from the requested host, so any future requests will be omitted, resulting in a speed-up. It may indeed result in frowny faces here and there, but the speed increase is worth it.
I have implemented such an approach using bind (so without Pi-Hole) and it works great. No frowny faces (but I guess that's a Chrome thing), just empty space where ads would've been.
Have you measured the speed increase? There is a speed increase with just normal Pi-hole anyways, since it's getting a very small response from a local server.
If you have the technical capability to set up bind then you are already way above where most of our users feel comfortable. I agree that NXDOMAIN is a faster way to handle things, but looking at the situation from our biggest user base, the home user that is running the install script on a Raspberry Pi, that level of sophistication just isn't needed. The biggest thread of the last few weeks on our Reddit sub has been how to deal with family members getting angry that their favorite sites are blocked. They need the ability to whitelist via webpage and to let their users know that sites are blocked because of Pi-hole. If they just returned a blank insert or no page at all the number of support requests for "The Pi-hole broke my internet" would skyrocket.
We have looked, and are again looking at how to modularize the code so that you can put in your own resolver, but with our dependency on dnsmasq for DHCP service, that will pretty much likely always be the default in an install, and the option would be for more sophisticated admins to adjust the settings to their comfort. There's only so much you can do with shell scripting and bash, and were already over 1100 lines of code just for the basic-installer.
I do appreciate the request, and this is the best place for the feature to be voted on and moved to a higher level of priority if needed, but with our extremely limited resources, we can only do so much so fast.
@DanSchaper I have to agree that if it happens frequently that "important" websites are blocked by Pi-Hole, it's better to return an informative message stating why it's blocked rather than just not getting any response.
However, I'm wondering: doesn't that mean that the default block list is too restrictive? In the Bind-configuration I mentioned above, I'm using the blocklist from MVPS. It blocks nearly all advertisements and trackers, but I never ever run into problems where sites do not work.
Anyway, I really appreciate the Pi-hole project even though I'm currently not using it. Lack of this feature is one of the reasons. It may be something you can add as an additional setting that is turned off by default, so that users can explicitly and deliberately turn it on when they desire the extra performance improvement it brings.
Can you specify what realistic performance improvement this would bring? Pi-hole itself improves web page performance by returning a very small placeholder instead of ads. I don't think this method would make such a performance boost that users would notice.
The frequency of seeing the Pi-hole block page is entirely up to the users browsing habits. Most of the devs run the default lists, and I for one have never seen the block list show up in the wild. Some users may use Google's shopping integration with search, which might cause them to be redirected through domains which are blocked, so that is when they'd whitelist that domain. Many users actually ADD lists, because they feel they need more domains blocked.
That assumes that the default block lists are the only lists enforced. We have users that often add in highly restrictive block lists and break things, with only the block page to let them know that they may be a little too aggressive in their actions. We have to balance the power users with the first timers and get a product that is fair and useful to everyone.
Being far on the power user side myself I would like to see us doing a lot more, but again we have very limited resources and far fewer users on the power end of the scale. This is a situation we are looking at how to best resolve, but swapping out the entire back end of the Pi-hole isn't something that is quite easily accomplished, while keeping the new user friendliness intact.
My apologies for digging up this quite old feature request (it's not closed yet !), but actually I find it interesting and upvoted it.
I totally understand that the "blocking page" feature is useful for most of users (even if with HTTPS almost everywhere it's becoming less and less useful) and PiHole should keep that feature by default.
But that would be useful to have the ability to configure PiHole so it sends NXDOMAIN for blocked queries, not much for performance increase (which would certainly be almost negligible as stated earlier) but to avoid having to set IPTables rules on PiHole to block TCP / UDP port 443 traffic (otherwise some websites got slow).
The UDP rule don't bother me, but the TCP rule prevents HTTPS hosting (or makes it unconvenient for the least).
OK, that concerns only a few people (me !), but even if you forget about HTTPS hosting, you have to admit it would be "cleaner" to be able to send NXDOMAIN (or even a custom IP for those wanting to have their own blockpage ?) instead of replying with PiHole IP, then generating a HTTPS connexion on the client side that will be blocked by Pi's IPTables.
Don't know what amount of work is required, but if such a feature was added (without modifying PiHole's default behavior), it could be useful to a few users (but I totally understand "a few" is no priority !)
I've just tried the "BLOCKINGMODE=NXDOMAIN" setting and so far everything seems to be doing fine : domains from blocklists are still blocked, and DNS replies sent by PiHole for those domains are "NXDOMAIN" as expected, very nice, thanks !
