Apologies in advance for this long post, but I just wanted to report back what I've learned trying to dig further into this issue, and wanted to be thorough. I'm not sure how to best categorize it (DNS vs DHCP issue), but it essentially seems to boil down to this (for Asus routers running Asus firmware), the router is essentially acting as a DNS "proxy".
What I found out - well lots of stuff, but here's the thing I believe to be the most relevant.
RMerlin (developer of Asuswrt-Merlin custom firmware for Asus routers) posted in another forum (I'm still new here, and don't know if linking to other forums is frowned upon, so I just copied his post from the other page. Also note that this post was from 2013, but it seems to accurately describe how the Asus routers are working today, as far as DNS is concerned.)
Asuswrt has the router run as a DNS proxy (that's the short version of it). That field on the DHCP page will usually contain your router's IP (that's the default when it's left empty), which means your clients will get your router's IP as the DNS, and then the router performs all DNS lookups using the DNS obtained either from your ISP, or manually entered on the WAN page. Having this means that your router can easily act as a caching server for DNS queries for your whole LAN, which can improve performances.
So if you wanted to use custom DNS servers, you would have to enter them on the WAN page, while leaving the DNS entry on the DHCP page untouched.
So between my testing and what I've learned about Asus routers, I believe the following to accurately describe the situation.
- The router will assign DNS Servers to all clients that obtain their ip address from the router's DHCP server. It will always assign its own ip address as one of the DNS servers.
- If the LAN - DHCP Server page has a DNS Server address, that address will be assigned as the first DNS to its DHCP clients, but it will still always assign itself as the secondary. The pihole's ip address should be assigned here to make sure it is given as the first DNS server to DHCP clients.
- The router is apparently acting as a DNS "proxy" and will use the DNS servers that are defined on the WAN - Internet Connection page, so the pihole needs to be defined here to ensure that requests made to the router are resolved by the pihole.
My own testing seems to indicate that if there is only the pihole address in DNS1 on the WAN page, and the pihole is not running, the router cannot resolve addresses because it simply doesn't know any other DNS servers unless DDNS Client is turned on, in which case, the built-in DNS "proxy" seems to fall back to the DDNS client for resolving DNS requests.
So from what I've read, it appears that it shouldn't matter that the router is giving its own ip address as the secondary DNS, since, as long as you've pointed DNS1 on the WAN page to the pihole's address, that is the address the router will use to resolve the requests. If I'm understanding this all correctly, then it seems we shouldn't even have to define the pihole ip address on the LAN - DHCP Server page at all, and clients can apparently use only the router's ip for DNS, and as long as the WAN DNS1 points to the pihole ip and there is no secondary address (and DDNS Client is disabled), then the router is going to use only the pihole to resolve DNS requests, and apparently will even cache them for faster serving to clients. Although I would still personally recommend assigning the pihole DNS to the DHCP page so clients query the pihole directly first, and indirectly (through the router) second.
My earlier tests seem to confirm this behavior, though I haven't yet attempted removing the pihole ip from the LAN - DHCP Server page (and leaving it blank there), and trying to just let the router assign itself as the DNS server to all the clients. But my testing does seem to indicate that if the pihole is the only address the Asus DNS "proxy" knows about, then that is the address it will use for resolving DNS requests.
If I am understanding how this Asus DNS stuff works, then I can simply add another pihole on the network and point the router's WAN DNS2 to the new one and that would provide redundancy if one is down.
I have to assume at this point, that the correct(?) way to configure the Asus routers to ensure that all traffic is going through the pihole is to:
- Set the DHCP DNS address on the router to the pihole's ip so clients will get that as their primary DNS. This is to ensure the clients at least try to resolve through the pihole directly first.
- Set the WAN DNS1 address to the pihole's ip so clients that weren't able to resolve through the pihole directly will 'fall back' to the router's DNS, which is actually getting its requests resolved by the pihole.
- You can leave the WAN DNS2 blank, or assign it to another pihole on the network. I currently have 184.108.40.206 to ensure connectivity if the pihole is down. If left blank and the pihole is not running, you will not be able to get online.
- Turn off the DDNS Client (WAN - DDNS page)
Clients should show the pihole's ip address as its first DNS and the router's ip as its second.
Here's what I believe is going on -
A client makes a DNS request, it first asks the pihole for the resolution. If, for whatever reason, the pihole doesn't respond or whatever, the client will fall back to the using router's DNS which is asking the pihole to resolve. This behavior can be observed in the logs, as occasionally, when a client makes a request, 2 identical requests show up, with 1 direct from the client ip and 1 from the router ip.
In other words, (if I've understood this all correctly) this configuration actually gives the client a "second chance" at resolving through the pihole since the router is using pihole to resolve. Then, if neither of those resolve, the router's secondary DNS is used (if an address is provided), and finally, if none of those resolve, the router will try through the DDNS Client (if it's enabled.)
I know there's still a chance something might connect through 220.127.116.11 as long as I have it listed there as WAN DNS2, but with all traffic having essentially 2 paths to the pihole, it doesn't appear that anything is getting through to connected clients from 18.104.22.168, at least not yet, and if I do go ahead with a secondary pihole, then I would be comfortable finally getting rid of that 22.214.171.124 once and for all!
So, as I've mentioned, I'm new to all this, and now it's time for me to ask, am I understanding this correctly? Or am I completely upside-down on this?
(Edit - ran one more test)
I ran one more test,
$ nslookup pi.hole <router_ip>
$ nslookup pi.hole <pihole_ip>
Both resolved correctly.
(Edit 2 - fixed a couple of typos)
(Edit 3 - Additional testing)
I did a 'spam' test of
$ nslookup pi.hole <router_ip>
where I just kept running the command and viewing the results.
With 126.96.36.199 as WAN DNS2, sometimes it resolved, sometimes it didn't, which is the expected behavior as the router bounces back and forth between 188.8.131.52 and the pihole. Removing that entry from DNS2, and the command always resolves.