How does using multiple levels of filtering/blacklisting affect performance? For example, I have OpenDNS with filtering, keyword blocking in Netgear router, and Pi-Hole, plus Pi-Hole manual blacklists (and whitelists). It seems like I might be making things more difficult for the network traffic, and I'd like to make things as efficient as possible while still retaining my filtering preferences. Thanks. Not urgent, just curious.
@peic Not exactly sure how to answer this or what you were expecting to hear, but all levels of "blocking" will have some impact on data throughput performance ... some, more than others.
One way to get an accurate picture would be to perform a data capture to a particular site with no form of blocking enabled to set a base line, then add each blocking element, one at a time, and then compare the results against the baseline. Wireshark would be a good tool for this.
Beyond throughput, one should also measure latency. This directly affects how "responsive" something feels.
I am using Neustar for my DNS resolver and their family filtering IP addresses. That covers me on the porn, phishing and other shady websites. Pihole just goes for the sites in my blocklists which could be equal to or greater than what Neustar filters out. So to me its just an extra later of protection and I reallyh dont see any latency or performance hits.
Thanks. I was just wondering if I should consolidate/streamline rather than have multiple filters. It's been somewhat of a shotgun method to try to block those results that OpenDNS fails to filter.
I presume your netgear router is set up to refer to the OpenDNS cloud based service? if so then from where I'm sited I see:
shoka@ShokaLaptop:~$ ping 208.67.222.222
PING 208.67.222.222 (208.67.222.222) 56(84) bytes of data.
64 bytes from 208.67.222.222: icmp_seq=1 ttl=55 time=17.1 ms <<<<<<<
64 bytes from 208.67.222.222: icmp_seq=2 ttl=55 time=17.1 ms
64 bytes from 208.67.222.222: icmp_seq=3 ttl=55 time=16.8 ms
^C
--- 208.67.222.222 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 16.894/17.054/17.145/0.188 ms
shoka@ShokaLaptop:~$ dig @208.67.222.222 google.com
; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> @208.67.222.222 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18886
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;google.com. IN A
;; ANSWER SECTION:
google.com. 300 IN A 172.217.168.238
;; Query time: **24 msec** <<<<<<<
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Sun Jan 05 15:34:57 GMT 2020
;; MSG SIZE rcvd: 55
Compared to say Google:
shoka@ShokaLaptop:~$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=54 time=`17.4 ms`<<<<
64 bytes from 8.8.8.8: icmp_seq=2 ttl=54 time=16.8 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=54 time=16.6 ms
^C
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 16.638/16.985/17.427/0.329 ms
shoka@ShokaLaptop:~$ dig @8.8.8.8 google.com
; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> @8.8.8.8 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60072
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;google.com. IN A
;; ANSWER SECTION:
google.com. 241 IN A 216.58.204.238
;; Query time: **17 msec** <<<<<
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Jan 05 15:40:17 GMT 2020
;; MSG SIZE rcvd: 55
shoka@ShokaLaptop:~
Presumably you want to use a recursive server as the back end to your DNS setup, unless you want to set up your pihole-FTL on your pihole server as a recursive server. Fair amount of work to do that.
So very little difference between the two services, so you may as well use the one that does filtering for you.
Again presumably your pihole is using OpenDNS as its upstream DNS server? If so it's accessing OpenDNS directly, not referring name requests to your router.
Your pihole is acting as a caching server so the latency you see will only be in the first request for a name, subsequent requests will be served by the pihole directly.
The latency to the pihole will be much smaller than to OpenDNS, as it's on your local LAN (presumably), thus adding the pihole in the path will speed up most requests and trivially, slow down (one additional LAN hop) the first request for a name. And only then if the client asked OpenDNS directly, rather than referring to your router to generate the name request.
in my case, first ask:
; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> @172.25.25.146 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42923
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;google.com. IN A
;; ANSWER SECTION:
google.com. 300 IN A 172.217.168.238
;; Query time: `26 msec`<<<<
;; SERVER: 172.25.25.146#53(172.25.25.146)
;; WHEN: Sun Jan 05 16:05:49 GMT 2020
;; MSG SIZE rcvd: 55
Second ask 
shoka@ShokaLaptop:~$ dig @172.25.25.146 google.com
; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> @172.25.25.146 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33512
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;google.com. IN A
;; ANSWER SECTION:
google.com. 293 IN A 172.217.168.238
;; Query time: `3 msec` <<<<
;; SERVER: 172.25.25.146#53(172.25.25.146)
;; WHEN: Sun Jan 05 16:05:56 GMT 2020
;; MSG SIZE rcvd: 55
shoka@ShokaLaptop:~$
I very much doubt that any private network will generate enough requests to swamp the pihole server, so as far as I can see adding the pihole into the mix is an unequivocal gain.
Harry
2 Likes
Really good explanation. I have found very little latency difference between the different upstream services like OPEN DNS, NEUSTAR, GOOGLE, etc...and 10msec or less probably makes no difference at all. Now if my local ISP (Comcast/Xfinity) would do filtering and would provide additional DNS IP addresses, that would bring my times down a few more milliseconds (Like I would really notice anyway).
Difference will be small, and you have to trust your ISP not to redirect your DNS requests to something favoring them, not always a safe assumption, as the present squeaking from some ISP's about the spread of DNSSEC, (which blocks those hi-jinks).
Harry
Just leaving a note here as I think it's worth mentioning: If you expect your ISP tinkering around and sending you IP addresses, you can flag them using the config option bogus-nxdomain=... in /etc/dnsmasq.d/99-custom.conf (or similar).
Transform replies which contain the IP address given into "No such domain" replies. This is intended to counteract a devious move made by Verisign in September 2003 when they started returning the address of an advertising web page in response to queries for unregistered names, instead of the correct NXDOMAIN response. This option tells dnsmasq to fake the correct response when it sees this behaviour. As at Sept 2003 the IP address being returned by Verisign is 64.94.110.11
We've seen this before also with Telekom, sending you to some ads-filled page with options of what you might actually wanted to visit. I think they stopped this practice, however, I have stopped using their DNS servers a long time ago, so ¯\_(ツ)_/¯
With this, you could also stop blocking pages from OpenDNS to show up.
2 Likes
Getting far off topic, but I've moved to using a local unbound instance configured for a recursive resolver with DNSSEC as the back end to my local DNS service. Next phase is to move the DNS traffic onto a VPN link, possibly a cross border VPN going to the Netherlands, since the UK government's lame and thankfully abandoned attempt at a porn blocking mandated such behavior from UK ISP's.
This is really good, and thorough. I do not understand some of it, but this comment was insightful:
"Again presumably your pihole is using OpenDNS as its upstream DNS server? If so it’s accessing OpenDNS directly, not referring name requests to your router."
Yes, I use OpenDNS (filtered) as my upstream, but I still have keyword blocking enabled in my router. Sometimes it does kick in, so I have to assume the traffic goes through the router at some level. Is that what you meant?
OpenDNS is a cloud based service, so has cloud scale latency. If you also have a filtering service running on your router that may increase your latency, because of the additional processing in the router but unless the router is very very slow it's latency will be swamped by the latency to the cloud.
If your router also has some DNS caching capability, it will also mitigate the cloud latency for repeat name queries, so making the gains form having a local DNS cache in the pihole, somewhat less impressive.
You can test things using utilities like dig or nslookup. Not sure what your clients use, but I've used nslookup on windows, so that certainly exists.
dig @ipaddress will direct your query to a specific server. The problem is that if that server does not have the name in its cache, it will recurse (ask upstream in effect), so you can be mislead about how quickly the server you think you are asking is responding.
You can flag dig not to recurse, so you will get a fast serverfail rather than a slow name resolve, to test if that is happening.
You can control what upstream service you direct your pihole to, but if you rely on your routers block list, then you need to keep it in the name lookup path.
Measure and see
Harry
1 Like