Benchmarking Pihole : Pi Zero vs Pi 3b+

General
1

Here's a benchmark comparing pi-hole running on a Pi Zero (with USB ethernet) vs a Pi 3b+.

tl;dr There was negligible performance difference for blocked domains, but a measurable difference in mean for forwarded + cacheable domains. Although the Pi 3b+ has a 11ms better mean response time for forwarded queries, the P95 for pi zero is better in both blocked and forwarded queries.

I would recommend using the Pi Zero.

Hypothesis

Prior to the experiment, I assumed that the pi zero would be 30-50% slower in all cases, and that stddev would be larger (worse & more erratic latencies).

Hardware & Setup

Router: TP Link Archer C9
PI 3b+ : connected via 100mb ethernet into router, 32gb sandisk microsd card. Raspbian Desktop
Pi Zero-W : connected via usb ethernet adapter, 100mb ethernet into router, 8gb class-4 sd card. Raspbian Lite

Method

I used dnstrace -- a go dns benchmark.
The client was a macbook pro connected via wifi.
Each test did 1000 queries with 10 concurrency ( go run dnstrace.go -c 10 -s 192.168.0.201 -n 100 www.googletagservices.com )

Results

full histogram and results can be found here

BLOCKED Domain www.doubleclick.net

pi 3b+

 min:		 16.252928ms
	 mean:		 38.372412ms
	 [+/-sd]:	 23.097544ms
	 max:		 385.875967ms

pi zero

DNS timings, 1000 datapoints
	 min:		 19.922944ms
	 mean:		 40.043544ms
	 [+/-sd]:	 7.762346ms
	 max:		 71.303167ms

FORWARDED domain www.amazon.com

pi 3b+

DNS timings, 997 datapoints
	 min:		 13.1072ms
	 mean:		 19.527493ms
	 [+/-sd]:	 12.101107ms
	 max:		 142.606335ms

pi zero

DNS timings, 999 datapoints
	 min:		 16.252928ms
	 mean:		 30.097752ms
	 [+/-sd]:	 5.005948ms
	 max:		 88.080383ms

Summary and Findings

I believe the zero is an adequate replacement for the pi 3b+ . Moreover, I was impressed and shocked that stddev on the pi zero was lower than the 3b+. So despite having worse mean forwarding response times, the P95 for pi zero is better in both blocked and forwarded queries.

:star: I'll leave these two devices online for a bit let me know if you'd like any further testing done. :star:

Updates

I've added benchmarks from a wired client to the gist results. The findings above are from the original wireless client.

2

i am quite interested in your 100gb ethernet. And surprise TP link router would support 100gb link. :open_mouth:

1 Like
3

haha me too i've fixed that typo :smiley:

4

I may not understand your test. A blocked domain should return from Pi-Hole in 1-2 msec at most. But you show means of 38 and 40 msec, with mins of 16 and 19 msec.

What is the time shown if you run dig www.doubleclick.net | grep Query

5 
$ dig @pihole3bplus.local www.doubleclick.net | grep Query
;; Query time: 2 msec
$ dig @piholezero.local www.doubleclick.net | grep Query
;; Query time: 3 msec

These individual responses are very fast. The results above are benchmarks to simulate bulk traffic (1000 queries x 10 concurrent)

6

Also, dnstrace seems to count network roundtrip, and I believe dig query time is not

7

I agree. When I run the same dig from my Mac terminal (wireless to the router, then to the switch and to the wired 3B+), it's 33-40 msec.

You may be testing your network more than either of the Pi-Holes. When I run this dig from the Mac to any of my Pi-Holes (3B+ wired or Zero W wireless), the query times are very similar. Within a msec. Averaging about 40 msec from the Mac, and 1 msec from the Pi, so that's 39 msec of network latency to any of them.

1 Like
8

Good point. I've added 4 more results taken from a WIRED client to the results gists. In that case servers and clients are all wired ethernet. The client was another pi zero

results GIST

9

And what do you conclude from this new data?

10

Overall still very impressed with the pi zero--would definitely recommend it. The wired results are a bit more as I would have expected -- the mean times for the 3b+ are all better. But the pihole stddev is very good which keeps it competitive.

How about you?

11

I have noticed no difference in DNS resolution speed between a wired 3B+ and a wireless Zero W in everyday use with a variety of client devices. So, from a purely Pi-Hole perspective, they run equally well and the platform difference is transparent to the user.

However, when you ssh into them, the 3B+ is significantly more responsive. Updates happen more quickly, rebuilding gravity (which you would never notice anyway, since it's done early Sunday morning normally), querying logs or databases.

A Zero W is plenty for Pi-Hole and at a fraction of the cost of a 3B+, a good choice.

1 Like