Hello all,
Currently I have pi-hole querying 8 local stubbys that are listening on 8 separate local ports doing dns over tls. I then have the stubbys piped through 8 separate tors. I can easily accomplish adding all of the extra local nameservers(server= entries) to a separate dnsmasq.d file and it happily makes it's way and works well. But, I was thinking it would be nice to be able to have a simple "add another dns server" selection in pi-hole to be able to add as many as are desired instead of being limited to the 2 custom servers as it is now and having to add more in a separate file.
The second small request would be addition of a check box for the enabling or diabling of the dnsmasq "all-servers" config option. This can accelerate lookups by querying all servers configured for each request and taking the first to answer instead of the default behavior of choosing just one to query in a round robin fashion.
The latter request is especially important for acceleration in my application by pumping 8 dns queries simultaneously over 8 tors and taking the fastest one to come back to help offset latent tor routes. As stated, It works well and I have it all configured and working with the extra dnsmasq.d file but would be nice to be able to do it through the web interface. I feel both requests are pertinent and would add new and applicable features for pi-hole's overall versatility, long term scalability, and speed.
Thanks for reading, really enjoying working with pi-hole!
Very interested in your solution, wondering how this works.
I have phole + unbound, without all-servers. Unbound is configured to listen on server=127.10.10.2#5552 and server=fdaa:bbcc:ddee:2::5552#5552
I don't know why, but dnsmasq appears to prefer the IPv6 resolver, even though I have configured only the pi's IPv4 address as DNS server, using DHCP, on all the clients, some, if not most, devices don't even have an IPv6 address.
dashboard / Queries answered by:
- blocklist 27.3%
- cache 11.9%
- unbound-ipv6 56.9%
- unbound-ipv4 4 4%
Looking at the FTL database (updated installation instructions in my installation manual, chapter 20, section 3 - the referred topic is for an older version), I see the results in the queries table:
I wonder if:
- When using
all-servers, are the requests to all the resolvers in thequeries table, or just the one served to the client? This would also be visible in thepihole query log(uses that table) - How does your `dashboard / Queries answered by graph look like? All resolvers same percentage or actually reflecting the fasted response?
@DL6ER: If this feature request makes it into pihole, obviously the query table (dashboard) could use an additional column answered by (the IP or name of the resolver), the info is already available in the database.
Uhm, why?
in reply to @jpgpi250
the requests to all servers are not showing up in the queries table as dnsmasq only accepts the fastest reply and drops the rest. I am showing 8 queries going outbound in the logs and a single reply, that single reply is what is then sent to client and reflected in my query log.
root@94ed689e0c35:~# grep testingtesting.com /var/log/pihole.log
Nov 8 10:35:28 dnsmasq[18893]: query[A] testingtesting.com from 10.8.0.6
Nov 8 10:35:28 dnsmasq[18893]: forwarded testingtesting.com to 127.0.0.1
Nov 8 10:35:28 dnsmasq[18893]: forwarded testingtesting.com to 127.0.0.1
Nov 8 10:35:28 dnsmasq[18893]: forwarded testingtesting.com to 127.0.0.1
Nov 8 10:35:28 dnsmasq[18893]: forwarded testingtesting.com to 127.0.0.1
Nov 8 10:35:28 dnsmasq[18893]: forwarded testingtesting.com to 127.0.0.1
Nov 8 10:35:28 dnsmasq[18893]: forwarded testingtesting.com to 127.0.0.1
Nov 8 10:35:28 dnsmasq[18893]: forwarded testingtesting.com to 127.0.0.1
Nov 8 10:35:28 dnsmasq[18893]: forwarded testingtesting.com to 127.0.0.1
Nov 8 10:35:28 dnsmasq[18893]: reply testingtesting.com is 91.195.240.126
Hopefully that answers your question? Let me know if you need any more info.
Well, honestly, I don't think this is a "good" feature as it drastically increases traffic. I see why you're using it in the use case you have, but I'm not yet convinced that an easily accessible switch would be (a) needed and (b) advantageous for most/many users.
Neither, nor. It will be first request made. Regardless, if this one request resulted in the answer finally served to the client. This is a mere optimization decision and a certain inaccuracy, I agree. However, this is still correct in all non-all-servers scenarios and difficult to implement, otherwise. The main difficulty is in that you probably don't want to see the very same query eight times being sent to different servers while it does not need to be clear who replies first at the time this particular query is written to the database. This is especially true for slow/unreliable connections. I don't see it being justified to implement a new UPDATE queries SET forward = '...' WHERE id = ... kind of thing. Especially also because we'd have to keep track of what was already updated and what not + what ID a query was added as to the database.
Note that database access is handled completely (and only) asynchronously in a non-blocking manner so we cannot simply "change it once" when we see that a reply made it through.
@DL6ER Thanks for the reply. The all-servers feature is simply a feature of dnsmasq. It's there for a reason, because people use it, and it works for specific scenarios, as I know you already know. If it isn't integrated into the gui, it's not a huge deal to me, I can still use it by creating a dnsmasq.d file. I just thought it would be an easy addition for what can be a useful feature. Lets get the focus off of the all-servers config to the more desired "add another dns server" for as many as the user wants to configure(this is simply supporting what dnsmasq already offers again) and is useful in many environments, for many purposes, does not add more traffic or do anything else that could be considered unsavory. I know this would be useful at least for advanced users. A passing thought... perhaps an "advanced options" area to enable these features for systems admins who know what they are doing and why they are doing it. 
Once again, it's ok if these don't get added, it's easy to put in the dnsmasq.d file. but it would be nice to be able to do it from the gui since these are all features of the software that pi-hole utilizes. One other thought for a feature request would be a toggle between successive or round-robin on the configured dns servers. Thanks again for the reply @DL6ER
hey there,
long time no see...
have you considered all-servers option in the aspect of increasing speed (drastically).
there's was this guy investigating the dnsmasq feature:
https://ma.ttwagner.com/make-dns-fly-with-dnsmasq-all-servers/
i use the all-server option since the very early versions of pihole with very good results. but adding the info of the fastet responding dns server would enable a local rank-statisitic on the set of ones personal server set.
from my point of view the info, which server provides the finally used reply is a valueable feature, also in the case, where redunancy is the use-case for multiple servers.
what do you think? adding "just" the single ip of the reply used for pihole in the dnsmasq output formatter?
Pi-hole already measures the speed of the configured DNS servers from time to time (I read this somewhere). What you are asking for
and I agree that it is not a useful feature in almost all cases. Personally, I don't see a difference in 28ms or 41ms for a reply. As said, asking more servers also means reduced privacy.
Maybe we can ask Mr. @PromoFaux. He seems wise.
This probably says it better than I could:
The DNS forward destination determination algorithm in FTL DNS's is modified to be much less restrictive than the original algorithm in
dnsmasq. We keep using the fastest responding server now for 1000 queries or 10 minutes (whatever happens earlier) instead of 50 queries or 10 seconds (default values indnsmasq). We keep the exceptions, i.e., we try all possible forward destinations ifSERVFAILorREFUSEDis received or if a timeout occurs. Overall, this change has proven to greatly reduce the number of actually performed queries in typical Pi-hole environments. It may even be understood as being preferential in terms of privacy (as we send queries much less often to all servers). This has been implemented in commit d1c163e.
Ok, I got the arguments: overhead & privacy, if the feature is not used wisely.
But, adding always the IP of just the first server in the config to all entries in the db, is not a plus to just write "outbound dns server". or is there any variation possible from blocklist/cache/one ip in the statistics of queries?
This is not happening following
This reply was for when all-servers is used. Yes, this is less meaningful. I guess the database would need to get restructured to be able to store an array of upstream servers.
During normal operation, the "best" DNS server is picked following the description @PromoFaux linked. When this algorithm found out that the third DNS server you have in your config is the fastest, then this will be used. This is not the same as
Just as a follow up, I'm the OP and I've been running pihole with dnsmasq configured with 'all-servers' querying 8 Stubbys over 8 Tor circuits for 6 months now as my main resolver on a mid sized VPN(I run all of this in a single docker container along with 2 Squids and 8 Privoxies). The amount of traffic generated by the extra queries has been near negligible. The performance improvement is significant when using DNS over TLS(stubby) through Tor circuits which would be considered a medium to high latency environment. This is now based on empirical data. Not speculation. For high Anon based networks that are fighting latency as a convenience and usability factor this is a great solution. Hopefully it will be implemented cleanly into pihole some day instead of having to use my dnsmasq.d file method. Isn't privacy, security, and usability what this project is supposed to be about after all?.... 
So when you have stubby running locally on your Pi-hole for doing DoT, why do you need to specify multiple upstream servers in Pi-hole? I would assume you only specify the one stubby you have locally (like on 127.0.0.1#5353) and this one is doing the requests to all the DoT servers out there?
In the environment I have built 8 Stubby's are running, each wrapped in proxychains4, each one running with it's own config that points to a unique local Tor SOCKS5 proxy, so 8 processes running that look like this, each with their own proxychains4 config to pass to a different, unique Tor circuit of which I have 8 running locally:
sudo -u stubby /usr/bin/proxychains4 -f /etc/proxychains-8.conf /usr/bin/stubby -C /etc/stubby/stubby-8.yml
example proxychains4 config(each of 8 configs looks like this but with a different local Tor proxy port defined:
socks5 127.0.0.1 9750
8 configs, each passes Stubbys encrypted DNS queries through 8 different(unique) local Tor circuits running as SOCKS 5 proxies locally. This creates a high-anon DNS environement on the frontend of a VPN even when the extra queries are taken in to consideration as 'revealing' because you sit behind Tor, Pi-hole, and the VPN. This is why the speed advantage becomes key for my purposes. I send 8 requests over 8 circuits and the fastest back gets used. In this environment with the possibility of congested Tor circuits and higher latency DNS over TLS it is key to cut milliseconds off where ever possible.
Additional DNSMasq config so that pihole can do this trick is then:
root@94ed689e0c35:~# cat /etc/dnsmasq.d/02-more-servers.conf
server=127.0.0.1#8033
server=127.0.0.1#8043
server=127.0.0.1#8053
server=127.0.0.1#8063
server=127.0.0.1#8073
server=127.0.0.1#8083
all-servers
2 of the local stubbies are defined in the pihole web-ui config which is why 6 are defined in the additional dnsmasq.d file.
So the DNS data path in this environ is:
Client PC/Device -> VPN ->DNSMasq(pi-hole) -> Stubby -> Proxychains4 -> Tor(SOCKS5) -> DNS SERVER
Caching, no DNS leaks, encryption, high-anon and as fast as it can be made given the inherent give-take relationship of speed versus privacy and security.
I then also have elite Squid proxies round-robining requests over the 8 Tors for high anon client browsing proxies. If you need any more details let me know. What you mentioned would certainly be a valid and good question in a different environment, hopefully this brings a little more light on to why this would be a nice feature added specifically for my purposes.
I think we can probably agree that the setup you have is fairly unique, and as such an edge case which is solved perfectly by manually adding a setting to the internal dnsmasq/pihole-FTL configuration.
If someone is willing to go this far, then adding a line to a config file instead of flipping a switch (that may not even be useful to most users) is not all that much extra work.
@PromoFaux Absolutely we can agree that my environment is not usual, which is why I made the statement about this being nice "specifically for my purposes" and why even in my first post I had said "it would be nice", but I have it covered if not. I'm fine with the dnsmasq.d file, it's caused no issues and is zero maintenance. I just thought it would be a nice small feature to add since dnsmasq supports this config, pi-hole is built on dnsmasq and I feel it's a gateway for people to explore more advanced and more anon dns configurations. That's all. So there's no argument, I'm just glad that everyone now actually understands what I'm doing in my environment and why this would applicable to it. Thanks for your time.