ISP DNS Cache/Proxy and how I bypass this nonsense

My ISP (Sky TV in the UK) does something rather irritating. They run a packet level filter on all traffic and intercept port 53 traffic in order to handle all DNS requests within their own system. You can test if your ISP does this by issuing this command.

nslookup google.com 45.45.45.45

Since 45.45.45.45 is not a public DNS server, the request should fail. If however it works and returns a result, then your ISP is caching DNS requests. This is REALLY bad because it negates the use of public DNS servers like 8.8.8.8 and causes latency on DDNS updates. There's privacy issues too, especially if the ISP is logging the requests (which it probably is). There's a whole host of other issues this can cause as well. Your ISP can resolve non-existent domains to their own IPs in order to show a custom "This Webpage does not exist" message in your browser. While that seems harmless at first, if you use DDNS systems a lot, this can cause huge issues when the software thinks a domain exists when it does not because it gets a DNS resolution from it. Trust me, it's a bad thing.

So anyway, enough with the rant. Let's get on to how I solved this.

First, I got myself a VPS (Virtual Private Server). I actually already had one, but I strongly recommend you get one. They are useful for a whole host of stuff. They are also quite inexpensive, mine only costs $5 a month. The place I use is called Digital Ocean, but that's just who I use, any VPS should do the trick. In my setup I also enabled IPv6 but that's optional. I also selected Ubuntu 18.04 as the operating system, so my tutorial here will be based on that. I should also mention that my Pi-Hole is also running on Ubuntu 18.04 which is on a thin client connected to my LAN.

Once logged into the VPS with SSH I went ahead and installed bind with this command

sudo apt install bind9

Once that was installed, I edited the config file with the following command

sudo nano /etc/bind/named.conf.options

There will be some stuff in here, just delete all of that and replace it with this

options {
        directory "/var/cache/bind";

        dnssec-validation auto;
        recursion yes;
        auth-nxdomain no;
        listen-on-v6 { any; };
        forwarders {
                8.8.8.8;
                8.8.4.4;
                };
        forward only;
        listen-on port 5555 { any; };
        query-source port 5555;
        allow-query { any; };
};

The listen-on-v6 line is only if you are using IPv6 of course. Obviously, you can change the forwarders to whatever you like and I would recommend changing the port from 5555 to any random port you want. Avoid using 5353 or other easily guessable port. And remember to open the port in the linux firewall. I won't go into that because different VPS providers use different ways of firewalling. They may have a network level firewall or may have to use linux iptables. I'll leave that up to you how you manage that. If your home IP is static it is worth only allowing access from your static IP. If it's dynamic, it'll probably always be in the same subnet, so only allow access from that subnet to limit the chances of your server being found and abused. I recommend using a firewall to block requests rather than using ACP because although ACP will deny requests it will expose that there is a service running. Also remember that if you're using IPv6 that there is a completely separate firewall for that, make sure you make the changes to both IPv4 and IPv6 firewalls.

Now restart the server

sudo service bind9 restart

Check it's working with

sudo service bind9 status

Now, that part of the setup is complete. Let's go over to Pi-Hole. Open your Pi-Hole Admin Console and select Settings on the left. Then select DNS at the top. Untick all the Upstream DNS servers on the left. On the right tick Custom 1 (IPv4) and enter the IP address of your VPS into the box followed by a # and the port number you chose. Assuming my VPS's IP is 66.77.88.99 and the port I used is 5555 I would enter

66.77.88.99#5555

Also, if you have IPv6 you should also tick Custom 3 (IPv6) and do the same, except use the IPv6 address followed a # and the port number. So assuming my v6 IP is 2a06:4433:1:a5::500a you would enter

2a06:4433:1:a5::500a#5555

Now scroll to the bottom and hit Save. That should be it.
Note: You may have to jump into the Pi-Hole server shell and restart it. I had to at least.

sudo service pihole-FTL restart

Job's done. No more ISP DNS caching.
Thanks to DanSchaper for showing me how to configure non-standard ports in Pi-Hole.

this should also be able to be fixed by using non standard ports. The key is the non-use of port 53. Perhaps finding a dns server that supports this.

check here.. https://www.reddit.com/r/HomeNetworking/comments/4f1nkk/bypass_dns_hijacking_transparent_dns_proxy_with/

True, however folk may be reluctant to use some random DNS server. I wanted to use Google's DNS so I needed to do it this way. Hopefully, the major Public DNS providers will start to offer non port 53 access at some point.

Have you tried running a local instance of a recursive resolver (i.e. unbound) to see if your ISP intercepts that traffic? Since unbound doesn't make a single DNS request to an upstream server, but splits the request into pieces directly to nameservers, the ISP might not be capturing/redirecting this traffic. And, if they are, then DNSSEC will not authenticate and the reply should be BOGUS.

I did come across a post somewhere about using unbound. It is something I am going to try. I'd never heard of it before so there's going to be a learning curve.

Easy 15 minute setup.

https://docs.pi-hole.net/guides/unbound/

1 Like

Even if they do so, the DNS traffic is apparent and ISP's will look for the traffic signature and not the port.

Just installed unbound and tried it out (you're right, the setup was easy). I'm getting

dig sigok.verteiltesysteme.net @127.0.0.1 -p 5353

; <<>> DiG 9.11.3-1ubuntu1.8-Ubuntu <<>> sigok.verteiltesysteme.net @127.0.0.1 -p 5353
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31156
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1472
;; QUESTION SECTION:
;sigok.verteiltesysteme.net.    IN      A

;; ANSWER SECTION:
sigok.verteiltesysteme.net. 1   IN      A       134.91.78.139

;; Query time: 66 msec
;; SERVER: 127.0.0.1#5353(127.0.0.1)
;; WHEN: Sun Aug 25 14:11:44 BST 2019
;; MSG SIZE  rcvd: 71

So it looks like my ISP is not messing with this. Not sure how DNSSEC works without doing some reading but I'm assuming there's no way they can spoof the signatures. So this is a local only solution, thanks. If the lag when accessing new domains is not an issue (which it isn't for me) then it's great.

DNSSEC is an authentication process. Very simple explanation (I'm sure I will hear from the experts) - the nameserver sends you an answer with an authentication hash. Using the other half of their private key, your computer calculates the hash and if they match, the reply has not been changed in transit.

This is Cloudflare's explanation of DNSSEC: How DNSSEC Works | Cloudflare

You will find that unbound has a very efficient cache, and as the cache populates then unbound has to do fewer inquiries to the name servers. It will also do cache-prefetching, where it populates the cache in the background before you ask for the information.

1 Like

Wow, Ubuntu 18.04's apt repository is hopelessly out of date with regard to unbound. I just spent hours trying to figure out why the TLS functions weren't working while trying to get a secure DoT setup running. Turns out, the installed unbound was 2+ years old. Looks like I'll have remove it then build/install the current version while working around the mess that "apt remove unbound" is going to leave behind. Looks like a few folk are maintaining apt repositories for it, but I never include repositories like that, they frequently break and cause headaches, plus I'm not comfortable with some rando's signing key.

In any case, rant over. Thanks for the heads-up on unbound. It seems to do practically anything DNS-wise you could ever wish to do.

EDIT: Giving up with this for now. While I can compile it fine, it refuses to run after install. The service is masked after install, after unmasking it and giving it a simple config it won't start.

> ā— unbound.service
>    Loaded: loaded (/etc/init.d/unbound; generated)
>    Active: failed (Result: protocol) since Tue 2019-08-27 09:15:58 BST; 3min 42s ago
>      Docs: man:systemd-sysv-generator(8)
>   Process: 24403 ExecStart=/etc/init.d/unbound start (code=exited, status=0/SUCCESS)
>  Main PID: 6140 (code=exited, status=0/SUCCESS)
> 
> Aug 27 09:15:58 wyze systemd[1]: Starting unbound.service...
> Aug 27 09:15:58 wyze systemd[1]: unbound.service: Can't open PID file /run/unbound.pid (yet?) after start: No such file or directory
> Aug 27 09:15:58 wyze systemd[1]: unbound.service: Failed with result 'protocol'.
> Aug 27 09:15:58 wyze systemd[1]: Failed to start unbound.service.

Also this:

$ sudo systemctl start unbound.service
Job for unbound.service failed because the service did not take the steps required by its unit configuration.
See "systemctl status unbound.service" and "journalctl -xe" for details.

trying to start it manually and

$ sudo unbound -c /etc/unbound/unbound.conf
[1566894477] unbound[24607:0] fatal error: user 'unbound' does not exist.

So I've uninstalled 1.9.2 and gone back to the repository version again. At least that one works.

If you are aiming for DoT don't count out stubby. I use it and IMO it is very stable and trustworthy.

edit: I build it from source it's pretty straightforward.

1 Like

Good call. Thanks.
After backing up my pi-hole.conf file to ~/

sudo apt remove unbound
sudo rm -rf /etc/unbound
wget http://cz.archive.ubuntu.com/ubuntu/pool/universe/u/unbound/unbound_1.9.0-2_amd64.deb
sudo dpkg --install unbound_1.9.0-2_amd64.deb
sudo apt install -f
sudo service unbound stop
sudo cp ~/pi-hole.conf /etc/unbound/unbound.conf
sudo service unbound start

ā— unbound.service - Unbound DNS server
   Loaded: loaded (/lib/systemd/system/unbound.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2019-08-28 05:49:42 BST; 2s ago
     Docs: man:unbound(8)
  Process: 32496 ExecStartPre=/usr/lib/unbound/package-helper root_trust_anchor_update (code=exited, status=0/SUCCESS)
  Process: 32493 ExecStartPre=/usr/lib/unbound/package-helper chroot_setup (code=exited, status=0/SUCCESS)
 Main PID: 32500 (unbound)
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/unbound.service
           ā””ā”€32500 /usr/sbin/unbound -d

Awesome. I'm also taking a look at stubby too. There's a lot of stuff in this field to check out. I ain't had so much fun in years. :grinning:

EDIT: All seems good so far with this conf file (some lines removed)

server:
    ...
    ... blah blah
    ...
    ssl-cert-bundle: "/etc/ssl/certs/ca-certificates.crt"

forward-zone:
    name: "."
    forward-tls-upstream: yes
    forward-addr: 2620:fe::fe@853#dns.quad9.net
    forward-addr: 9.9.9.9@853#dns.quad9.net
    forward-addr: 2620:fe::9@853#dns.quad9.net
    forward-addr: 149.112.112.112@853#dns.quad9.net
    forward-addr: 2606:4700:4700::1111@853#cloudflare-dns.com
    forward-addr: 1.1.1.1@853#cloudflare-dns.com
    forward-addr: 2606:4700:4700::1001@853#cloudflare-dns.com
    forward-addr: 1.0.0.1@853#cloudflare-dns.com

Using dig I can resolve host names fine.

$ dig -p 5353 @127.0.0.1 google.com
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64986

Then if I change the host names in the config by just adding a character to the end of them to test the cert auth
forward-addr: 2620:fe::fe@853#dns.quad9.net1
restart unbound and try to resolve a host

$ dig -p 5353 @127.0.0.1 google.com
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 29224

So looks like all is good.

Imgur

IPv6 passing all DNS tests too.

What advantages do you see running unbound as a forwarder? In exchange for encrypted DNS, you are trusting third party DNS services with your DNS history. Have you considered running unbound as a local recursive resolver?

I'm just trying everything. I'm learning.

1 Like