How to get rid of "ghost" DNS servers

[sarambas]

Hi there.
I am running Pi-hole on a Raspberry Pi 3 in a non-unusual setup. I have a Netgear Nighthawk RAX200 Wi-Fi 6 router that I've had since late 2020. I have been successfully using Pi-hole since December 2022 to block out ads, which has made browsing the web a much more pleasant experience. The router is the DHCP server, it has the Pi's wlan0 interface (used for Pi-hole) set to a static IPv4 address, and it has IPv6 enabled.

However, I noticed today that my ad blocking isn't working. It appears to be related to IPv6. I provide a value for primary (not secondary) DNS in the IPv6 section, yet all clients on my network report these additional DNS servers:

2001:558:feed::1
2001:558:feed::2

These appear to be the default IPv6 DNS servers for Comcast, my ISP.

I have no idea why these DNS servers are being propagated to each of my devices. I know these are defaults, but my assumption is that by setting a custom DNS server, it should override any ISP-provided DNS servers, and this matches behavior I was experiencing before today.

To isolate, I have even tried setting both primary and secondary IPv6 DNS servers to CloudFlare's and clearing out my Pi-hole's IPv6 DNS entirely, but both Comcast IPv6 DNS servers still come down to other devices.

Unfortunately, this problem effectively defeats Pi-hole entirely since the ads/trackers will find their way around using the Comcast IPv6 DNS servers.

I have tried contacting both Comcast and Netgear support. Both companies have proven equally frustrating and unhelpful.

I've already taken a look at similar issues, and it seems the only thing that has worked for these folks is to disable IPv6 altogether. However, this seems undesirable and only a last resort if the root cause of the extra DNS server propagation cannot be determined or fixed.

I am not well versed in IPv6 or the mechanics of what would cause default ISP DNS servers to be sent to clients even despite custom DNS being set.

Goals:

• To determine why the ISP DNS servers are being forced to clients
• Fix or eliminate this and force router to only use Pi-hole as DNS server (IPv6)
• To avoid disabling IPv6 if possible

Expected Behaviour:

Devices on my network inherit only DNS servers that I've custom defined in my router for both IPv4 and IPv6.

Actual Behaviour:

I receive additional unexpected IPv6 DNS servers on clients, thus defeating Pi-hole entirely.

Debug Token:

L3i3YT6X

Thanks for reading and I appreciate any help!

[Bucking_Horn]

@sarambas

2001:558:feed::1
2001:558:feed::2

These appear to be the default IPv6 DNS servers for Comcast, my ISP.

I have no idea why these DNS servers are being propagated to each of my devices.

Your router would be advertising those IPv6 address as DNS servers, allowing your clients to by-pass Pi-hole.

You'd have to find a way to configure your router to advertise your Pi-hole host machine's IPv6 as DNS server or to stop advertising any alternatives.

You'd have to consult your router's documentation sources on further details for its IPv6 configuration options.

If your router doesn't support configuring IPv6 DNS, you could consider disabling IPv6 altogether.

If your router doesn't support that either, your clients will always be able to bypass Pi-hole via IPv6.

[nosugref42]

My network has a Netgear RAX80 router, and I could never figure out how to get IPv6 properly configured. The router’s admin interface help is a joke, and frankly, I have had so many issues with configuring the admin interface, I trust the router to do almost nothing for me anymore.

I tried many settings for IPv6, and nothing appeared to work. I always saw my ISP’s IPv6 DNS. Initially, I figured it was me, and very well could be, but I decided to disable IPv6. Problem solved for me.

I also stopped relying on the router for DHCP. It has a frighteningly buggy interface in this area. Mine has been in place for a few years, and it would show old devices that no matter what I could not remove. Pretty sure even after a factory reset they were present.

Disable IPv6, and DHCP. Let Pi-hole handle DHCP, and my network has been solid. The Dashboard, and Query Log have reflected network activity cleanly for me since then.

[deHakkelaar]

@nosugref42

... , and it would show old devices that no matter what I could not remove.

I've heard before of an implementation where hostnames that are discovered via DHCP are stored/added in the local /etc/hosts file on the router.
But no maintenance is done on this /etc/hosts file when hostnames are changed.
Folks had to manually remove the entries from this file to get it fixed.

[sarambas]

This is unfortunate to read. For a several hundred dollar router, the administration interface is indeed disappointing and sloppy.

One thing to mention is that I did perform an upgrade to the latest router firmware last week ( V1.0.10.140_1.0.79) but then immediately went out of town for a few days. Only when I got back Sunday evening did I notice this DNS issue was occurring. So things were working just fine before, and I even recall checking the DNS section of various devices' network settings, and I did not recall seeing the Comcast provided servers. It's possible this latest firmware update introduced a regression to cause this problem (or worse, introduced this as a "feature").

I was trying to get by without disabling IPv6 and without disabling the router's DHCP functionality. I don't have where I read this handy currently, but I recall reading somewhere that the Pi's DHCP server is generally really intended only if your router won't let you change these settings out of the box. And anyway it seems like this should "just work".

@deHakkelaar

I've heard before of an implementation where hostnames that are discovered via DHCP are stored/added in the local /etc/hosts file on the router.
But no maintenance is done on this /etc/hosts file when hostnames are changed.
Folks had to manually remove the entries from this file to get it fixed.

This is an interesting concept. Do you know if these folks used custom firmware? If it comes to this, I am fine ssh'ing into the router and changing things myself if I can figure out how.

It seems at this point my options are:

• Disable IPv6 and use the Pi's DHCP server
• Try to haggle support out of Netgear (and hopefully avoid a $130 fee)
• Install custom firmware on the router

None of the above seem ideal, but at least the last one would potentially let me bypass the crappy stock UI and get more fine-grained control over things.

[deHakkelaar]

@sarambas

I don't have where I read this handy currently, but I recall reading somewhere that the Pi's DHCP server is generally really intended only if your router won't let you change these settings out of the box.

True but also for other purposes.
Bc dnsmasq is embedded into the pihole-FTL binary, it also allows all the features from dnsmasq to be used:

pi@ph5b:~ $ man dnsmasq
[..]
DESCRIPTION
       dnsmasq  is a lightweight DNS, TFTP, PXE, router advertisement and
       DHCP server. It is intended to provide coupled DNS and  DHCP  ser‐
       vice to a LAN.

This allows tons of tweaking capabilities that isnt possible on most consumer routers.

@sarambas

This is an interesting concept. Do you know if these folks used custom firmware?

I'll have to lookup.
Report back shortly.

If you need a handy tool to troubleshoot DNS servers advertised via IPv6 RA (Router Advertisement), you could install below on the Raspi:

sudo apt install ndisc6

And run below:

rdisc6 <NETWORK_INTERFACE_NAME>

Below shows no DNS servers advertised via IPv6 RA (or any RA adverised):

$ rdisc6 eth0
Soliciting ff02::2 (ff02::2) on eth0...
Timed out.
Timed out.
Timed out.
No response.

And below does:

$ rdisc6 eth0
[..]
 Prefix                   : fd00::/64
[..]
 Recursive DNS server     : fd00::3ea6:xxxx:xxxx:xxxx
[..]
Source link-layer address: 3C:A6:2F:XX:XX:XX

Above Source link-layer address can be looked up below to determine which device is advertising:

mac+address+lookup at DuckDuckGo

EDIT: Or through Pi-hole

pi@ph5b:~ $ pihole-FTL sqlite3 /etc/pihole/macvendor.db "SELECT vendor FROM macvendor WHERE mac LIKE '3C:A6:2F'"
AVM Audiovisuelles Marketing und Computersysteme GmbH

[deHakkelaar]

@deHakkelaar

@sarambas

This is an interesting concept. Do you know if these folks used custom firmware?

I'll have to lookup.
Report back shortly.

Below says a USG device stores double or old names in /etc/hosts:

https://gathering.tweakers.net/forum/list_message/65727160#65727160

EDIT: Ow USG = Unifi Security Gateway from Ubiquiti

[Bucking_Horn]

@sarambas

Disable IPv6 and use the Pi's DHCP server

Note that DHCP is strictly IPv4.
For IPv6, DHCPv6 would be roughly equivalent, but most client OSs would instead prefer to use SLAAC/NDP/RA to join and discover your network (and some OSs would not even support DHPCv6 at all, e.g. Android).

Thus, it would not be necessary to switch from your router's to your Pi-hole's DHCP server - unless your router would mangle DHCP and DHCPv6 settings into one configuration option (unfortunately, quite a few router models are not very clear in naming their IPv6 DNS configuration options, if they'd even expose them).

In any case, you should stay watchful:
Disabling DHCPv6 could still mean that your router advertises local DNS resolvers via NDP/RDNSS router advertisements. Those need to be configured to use one of your Pi-hole host machine's stable IPv6 addresses, or to be disabled as well.

@sarambas

@deHakkelaar

But no maintenance is done on this /etc/hosts file when hostnames are changed.
Folks had to manually remove the entries from this file to get it fixed.

This is an interesting concept. Do you know if these folks used custom firmware?

That exchange does not seem to be related to your topic's observation.
If it would become an issue, please consider opening a new topic for it, to keep things focused.

[sarambas]

@Bucking_Horn

Note that DHCP is strictly IPv4.
For IPv6, DHCPv6 would be roughly equivalent, but most client OSs would instead prefer to use SLAAC/NDP/RA to join and discover your network (and some OSs would not even support DHPCv6 at all, e.g. Android).

Thus, it would not be necessary to switch from your router's to your Pi-hole's DHCP server - unless your router would mangle DHCP and DHCPv6 settings into one configuration option (unfortunately, quite a few router models are not very clear in naming their IPv6 DNS configuration options, if they'd even expose them).

Thanks for the explanation @Bucking_Horn. This is where I have some learning to do about IPv6

I found this thread on a whim where the poster EG's first reply said, "Generally, you should use Auto-Config." Given this, I looked again in my router's interface in the IPv6 settings and opened the drop-down for "Internet Connection Type" seen below. Sure enough, there is an "Auto Config" option.

Previously, I was using "Auto Detect" and I had also tried "DHCP". I went ahead and tried the "Auto Config" and hit Apply. Voila! I checked my devices' DNS settings, and they now have only the IPv4 and IPv6 DNS resolvers I manually specified in the router config. Both of these point to the Pi-hole's host machine. The Comcast resolvers are gone.

Of course, I am not sure of the underlying mechanics here or what the difference between "Auto Detect" and "Auto Config" really means, but so far this seems to have resolved the issue for me. I'll report back if this changes.

@nosugref42 - perhaps this is something that the RAX80 has as well you could try?

I appreciate everyone's help, and hopefully someone else who comes along finds this thread helpful too.

[Bucking_Horn]

@sarambas

I looked again in my router's interface in the IPv6 settings and opened the drop-down for "Internet Connection Type" seen below.

It would seem -let's say- unexpected that changing an option for your router's external "IPv6 Internet Connection Type" would have an impact on your internal private network's DNS configuration. But that's also the reason why I can only defer to the router's documentation and support for its IPv6 DNS configuration - it's just too cryptic to guess what they mean, even with screenshots.

For now, I'll keep my fingers crossed that it does what you want.

[sarambas]

I agree it's unexpected! The manual doesn't seem to give a whole lot of detailed insight on how this works. https://www.downloads.netgear.com/files/GDC/RAX200/RAX200_UM_EN.pdf

[nosugref42]

@sarambas - thank you for that. My router does have that option available, and if I find a need for IPv6 support, I will check into it further. At the moment, IPv4 is sufficient for my network.