Pi-hole v5 v's v6 beta - milliseconds v's microseconds?

I have two Pi-holes setup both running on separate Raspberry Pi 3a+ on the same LAN acting as separate DNS resolvers with Unbound.

I thought I would try comparing (not very scientifically) the lookup times whilst browsing some websites on two different Xiaomi phones, one using Pi-hole v5 as it's DNS and the other using Pi-hole v6 beta as it's DNS.

The first attached image is the list of Domains which were resolved, which I verified are the same in both Pi-hole v5 & v6 beta.

The second image is the time (in brackets) for Pi-hole v5 to resolve the domains.

The third image is the time for Pi-hole v6 beta to resolve the same domains.

You can see that for the first domain (dpm.demdex.net) that Pi-hole v5 was quicker at 0.5ms, whereas Pi-hole v6 beta took 0.8ms.

Thereafter, Pi-hole v6 beta seems to shift to time measured in µs (ie. microseconds). Now, the last time I checked 1ms = 1000µs

So, Pi-hole v6 betas measurement of 66.5µs to resolve the second domain lookup, is orders-of-magnitude faster than v5's measurement of 0.9ms.

66.5µs = 0.0665ms, so that would mean Pi-hole v6 beta is approximately 13 times faster than Pi-hole v5's time of 0.9ms.

So, here's my question: is this right? Is Pi-hole v6 beta really 13 times faster at resolving domains than v5?

Obviously, it's not a very extensive test, but I'd be interested in others observations.



No, and this cannot be said from such single measurements. I know you have said that you didn't want to do it too scientifically but such numbers should still be sound when being discussed.

We have not done any such benchmarks as doing them thoroughly is complicated and doesn't provide much insight else than something you'd put something glossy selling magazine (and Pi-hole is not "sold" :wink: ). Please don't see this disparaging of anything you did here, just to explain why we have not put the necessary effort into it.

What makes this difficult is the very large parallelism going on in Pi-hole making measurements very difficult. DNS is an asynchronous protocol. Just let me give a rather simplified example:

  1. A client makes a query
  2. If you Pi-hole is idle at this very moment, it immediately processes the query,
  3. and sends a reply

This is the ideal case, but what may happen as well is:

  1. A client is makes a query
  2. your Pi-hole is currently busy with something else and picks up the query a tiny moment later (it is strictly FIFO). This already adds some unforseeable delay to the processing time, then while processing the reply it may happen that
  3. you are currently using the web interface and the API has to provide some numbers for which a consistent data base is necessary (e.g. while preparing the data for a page of the Query Log page). While this consistent state is active, no new queries can be processed, only thereafter
  4. the reply is generated and sent to the client.

It is very possible that the same process takes 60 µs while the second takes 6 ms although the same version of Pi-hole on the same machine - yet we see a factor of 100x difference just caused by external influences.

Okay, I appreciate that there could be all kinds of factors impacting how long each query takes and it's a bit like a traffic policeman directing multiple flows of traffic coming at him all at once.

But, in my observations, both Pi-holes were largely idle (each with only one or two clients), and consistently the Pi-hole v6 beta seems to resolve in microseconds compared to v5's milliseconds.

And yes, I guess some lookups on v6 beta took longer than v5 (like the very first one in my non-scientific test) which might eat-up all the other time savings. But my query was really, in very general terms, does v6 perform faster than v5? And, if nothing else were different, it would perhaps be worth switching over to v6 beta now & possibly observing an improvement in performance, and certainly not a worse performance?

Just wondering if it'd be worth taking some pain now in switching to v6 beta, and getting some payback in terms of better performance. Just a thought really.

Yeah, appreciate you think about this. In general, the beta is mostly done. We are currently trying to resolve two edge-case bug reports for which we are not even fully sure if they are bugs or if there is some continued misunderstanding between us and the original reporters. Both issues are very unlikely to affect anyone else and we have not been able to reproduce their finding using the exact same config on one of our testing devices.

So, you can switch over to the beta at any point, it is basically "done". We have thought about releasing before Christmas but then didn't because we wanted to have everything rock-solid instead. Granted, "everything" will never be perfect on software but we're close to the new official dnsmasq v2.91 release and we have all the code already in the beta.

TL;DR: When you switch to the beta, you will most likely see the final "product". Only minor stuff is still under tweaking these days.

It's probably "only days to weeks" but I have thought the same last month (and the one before), too - so no promises :wink:

1 Like