Unexplained Drop From 13% to 2-6% Block Rate

Please help!

Setup:
Raspberry Pi Zero W running Raspberry Pi OS Lite (Raspbian 10 Buster) on Linux 5.10.11+ kernel connected via wifi to TP-Link Archer 10 Pro router

I am located in Japan and my ISP (Docomo Hikari/OCN) provides the usual IPv4 as well as something they call V6Plus (which is apparently an IPv6 IPoE connection unique to Japan? I think it's similar to IPv4 over IPv6? I'm not really sure how it works exactly). I had attempted to install the pihole on my home network using the NEC wifi router provided by my ISP but it did not allow for setting the primary/secondary DNS on the router and other useful settings so I ended up purchasing a TP-Link Archer 10 Pro (a Japan-only model of the Archer 10 made to handle the aforementioned Japan-specific IPv6). I switched out the NEC router for the TP-Link router which allowed me to set up all the traffic to be redirected to the pihole from my router (without pointing each device to it manually) using primary/secondary DNS in the DHCP settings. However, this left my pihole from being able to connect to the new router (since the IP addresses had changed). I fixed this by popping the SD card into my linux system, manually editing the wpa_supplicant.conf file for the new SSID/pwd, and assigning the pihole a static IP in the new router's range from inside the router's admin panel. This allowed the pihole to work and blocked around 13% of all queries using the default list (including everything on speedtest.net). So far, so good but I wanted to get more out of my pihole and decided to try to implement DNS over HTTPS using cloudflared. Everything seemed to install and setup fine but did not work in practice. Not only did it not successfully implement DNS over HTTPS according to 1.1.1.1/help, my pihole was suddenly no longer blocking ads on speedtest.net. I also noticed some others ads were still getting through so I tried to add a bunch of recommended adlists to no avail. I don't now for sure whether this was the fault of a botched cloudflared install or not.

Either way, I got fed up and unplugged the Pi Zero W, wiped the SD card, and did a clean reinstall of the OS and pihole.This time it was connected to the new router from the start and all setup was done with my current network settings (no changing routers in the middle). Everything seemed to install and setup fine but, whereas the last installation of the pihole was blocking 13% of ads out of the box, this one was blocking 0% before a pihole -d revealed that neither of the IPv6 addresses bound to the wlan0 interface matched the setupVars.conf file. I manually edited the file to change the last four digits from ":1001" to ":1002" which caused the first of the two addresses to then match. Afterwards, the pihole began to block 2% of queries... better but still far from what had worked before with no tinkering.

As far as I can tell as a complete novice, everything else in the diagnostic report is coming up as it should be but I'm getting very poor results. After a day of use it's now blocking 6%. Any ideas what could be causing this or how I can fix it?

My debug token is: https://tricorder.pi-hole.net/rgci2lpqab

Not sure if It will be useful but an nslookup example.com returns:

Server:		1.1.1.1
Address:	1.1.1.1#53

Non-authoritative answer:
Name:	example.com
Address: 93.184.216.34
Name:	example.com
Address: 2606:2800:220:1:248:1893:25c8:1946

Thank you!

First, your blocking rate is expected to fluctuate, as it entirely depends upon the traffic that your clients generate, like your browsing habits on a PC, or an IoT device peaking on running its weekly updates or a "smart" TV showing a massive increase in DNS requests when streaming media while keeping a somewhat lower profile otherwise, etc.

By no means is a blocking rate an indication of Pi-hole's level of operation.
A blocking rate of 100% doesn't mean Pi-hole is 100% effective, but would rather give you the same browsing experience as pulling the plug on your router.
And then again, if none of your clients were using Pi-hole for DNS, your blocking rate would be zero, no matter the size of your blocklists.

A drop in blocking rates may indicate your clients may bypass Pi-hole sometimes, or some of your clients have stopped using it at all.
But just as well you may simply observe your network during a quiet period.
It's impossible to reliably assess this without having carefully scrutinised your past and present DNS traffic.
(I'm mentioning this more for the benefit of the casual reader observing fluctuations of blocking rate. That observation by itself is perfectly normal.)

Now that that's out of the way, you should focus your efforts on ensuring hat Pi-hole is the only DNS server for your clients.
Once that is working as expected, you may then consider adding additional blocklists or upstream servers like cloudflared for DoH.

Your desciption of your IPv6 address configuration might allow us to guess that your devices may bypass Pi-hole using IPv6, but your nslookup shows that the machine you have run that command from is not even using Pi-hole's IPv4 address as DNS server, but rather 1.1.1.1.

That's unexpected, as your DHCP server is correctly distributing Pi-hole's IPv4 as local DNS server:

*** [ DIAGNOSING ]: Discovering active DHCP servers (takes 10 seconds)
   Scanning all your interfaces for DHCP servers
   
   * Received 300 bytes from wlan0:192.168.0.1
     Server IP address: 192.168.0.1
     DHCP options:
      Message type: DHCPOFFER (2)
      lease-time: Infinite
      dns-server: 192.168.0.169
      dns-server: 192.168.0.169
      router: 192.168.0.1
      --- end of options ---

But note that your router is handing out a DHCP lease of Infinite validity.
If that has been the case before your changed DNS servers in your router, your clients would never request a new DHCP lease and update their information, unless you force them to do so (e.g. by power-cycling them).

If that nslookup was run from your Pi-hole machine instead, that would be ok then:
The Pi-hole host machine may use any DNS you prefer without interfering with Pi-hole's correct operation.

To verify your clients are using Pi-hole, please post the output of the following commands as run from a client machine:

nslookup pi.hole
nslookup flurry.com

Thank you for your reply!

Yeah, that makes perfect sense with regard to the blocking rate. In hindsight, that's a perfectly reasonable explanation I had not considered. Thank you for pointing that out.

Here is the output from the commands as run from my MacBook Pro on the same network:

 $ nslookup pi.hole
Server:		2400:4053:9421:500:2eff:65ff:fe44:4f11
Address:	2400:4053:9421:500:2eff:65ff:fe44:4f11#53

** server can't find pi.hole: NXDOMAIN
 $ nslookup flurry.com
Server:		2400:4053:9421:500:2eff:65ff:fe44:4f11
Address:	2400:4053:9421:500:2eff:65ff:fe44:4f11#53

Non-authoritative answer:
Name:	flurry.com
Address: 74.6.136.150
Name:	flurry.com
Address: 98.136.103.23
Name:	flurry.com
Address: 212.82.100.150

Also, in case it might prove useful, here are some screenshots of my router settings.

Originally, the pihole had shown up in my DHCP clients area and I selected it from there to reserve the IP address 192.168.0.169 for it. It then showed up there with the name "pihole". However, as you can see, it no longer shows up in the DHCP clients list and the previous reservation, while still working as far as assigning the proper IP address to the pihole, is now showing the device name as "UNKNOWN".

That's the IPv6 bypass we've guessed at.
That Macbook Pro is using an alternative DNS server via IPv6, bypassing Pi-hole. Thus it fails to resolve pi.hole (which is only known by Pi-hole) and doesn't block flurry.com.

You should configure your router to stop offering its own IPv6 address as DNS server, lest your devices will bypass Pi-hole via IPv6.

But let's not forget about IPv4 and check that as well.
I am not really fluent in Macs, but the following command should list DNS servers your Macbook Pro uses:

scutil --dns

c0dy ,
Just wondering, given you have a range of 100 - 249 why not use an ip outside the range for the pihole?

5 posts were split to a new topic: Pi-hole by-passed via IPv6 on Asus RT-AC66U-B1

If you opted to set a static IP address during Pi-hole's installation, your RPi will now be configured with a static IPv4 address via /etc/dhcpcd.conf. With such a static IP, your RPi won't request a DHCP lease anymore.

Since you've also configured a DHCP lease reservation for the same device and address in your router, you're successfully avoiding that your router would hand out the same IP to another device.

Though you could also have your RPi requesting a DHCP lease (by clearing entries from dhcpcd.conf) or to move your RPi's IPv4 outside of your router's DHCP pool as mentioned by nemoyi4714 (to avoid your router handing out the same IP), there's no immediate reason to change your configuration,
You can leave that as is right now.

If you decide to change it anyway, I'd prefer to go the DHCP way, as that would make switching routers less of a hassle, but that's a personal choice.

Ah, so it is related to the IPv6 after all... Hmm.

I also have a Linux desktop running Pop!_OS 20.04 on the network so I can do any diagnosing via that if it makes it easier (as opposed to Mac).

scutil --dns on the MacBook Pro returns:

$ scutil --dns
DNS configuration

resolver #1
  nameserver[0] : 2400:4053:9421:500:2eff:65ff:fe44:4f11
  nameserver[1] : 192.168.0.169
  nameserver[2] : 192.168.0.169
  if_index : 5 (en0)
  flags    : Request A records, Request AAAA records
  reach    : 0x00000002 (Reachable)

resolver #2
  domain   : local
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300000

resolver #3
  domain   : 254.169.in-addr.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300200

resolver #4
  domain   : 8.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300400

resolver #5
  domain   : 9.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300600

resolver #6
  domain   : a.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300800

resolver #7
  domain   : b.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 301000

DNS configuration (for scoped queries)

resolver #1
  nameserver[0] : 2400:4053:9421:500:2eff:65ff:fe44:4f11
  nameserver[1] : 192.168.0.169
  nameserver[2] : 192.168.0.169
  if_index : 5 (en0)
  flags    : Scoped, Request A records, Request AAAA records
  reach    : 0x00000002 (Reachable)

When you mention configuring the router to stop offering its own IPv6 address as DNS server, I attempted to do that on my own in the router settings on this page:

Before asking for help here, I had a feeling it was related to IPv6 and tried to fix it myself. On the [Advanced] tab, I tried changing "Get IPv6 Address" to DHCPv6 and changed "DNS Address" from "Get Dynamically from ISP" to "Use the Following DNS Addresses" and entered the IPv6 address given to me at the end of the PiHole setup for both primary and secondary DNS and saved. Instead of fixing anything though, it rendered every device on my network unable to connect to the internet.

Underneath the section in the screenshot above there is another section for LAN that looks like this:

Not sure if it is relevant, though.

During Pi-hole's installation, I chose to use existing network settings as I had already made the IP reservation for the device in the router beforehand. I did not manually edit the /etc/dhcpcd.conf for this installation.

EDIT:

I SSH'd into the pihole just now and ran a cat /etc/dhcpdb.conf and noticed that there was indeed an entry for interface wlan0 setting a static IP. I deleted this entry and rebooted the RPI. After that, I logged into the router admin page and removed the lease reservation, refreshed the DHCP Client List. Success! The pihole was once again showing up so I created another reservation for the pihole, this time with the IP 192.168.0.69 (making it outside the 100-249 range mentioned by nemoyi4714 above).

Now $nslookup example.com on the pihole returns:

pi@pihole:~ $ nslookup example.com
Server:		192.168.0.1
Address:	192.168.0.1#53

Non-authoritative answer:
Name:	example.com
Address: 93.184.216.34
Name:	example.com
Address: 2606:2800:220:1:248:1893:25c8:1946

Further, nslookup pi.hole and nslookup flurry.com on my Pop!_OS system now returns:

$ nslookup pi.hole    
Server:		127.0.0.53
Address:	127.0.0.53#53

** server can't find pi.hole: NXDOMAIN

$ nslookup flurry.com
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	flurry.com
Address: 98.136.103.23
Name:	flurry.com
Address: 74.6.136.150
Name:	flurry.com
Address: 212.82.100.150

EDIT2:

The above values seemed off to me and I could no longer login to the pihole admin page, so I attempted a pihole -r and reconfigured the pihole with the new settings.

The pihole now shows the IPv6 address of the pihole has been changed to end in 1004 and an nslookup example.com seems to now be showing the pihole's IP as server instead of the router's.

pi@pihole:~ $ hostname -I
192.168.0.69 2400:4053:9421:500:c2c9::1004 
pi@pihole:~ $ nslookup example.com
Server:		192.168.0.69
Address:	192.168.0.69#53

Non-authoritative answer:
Name:	example.com
Address: 93.184.216.34
Name:	example.com
Address: 2606:2800:220:1:248:1893:25c8:1946

Additionally, on my Pop!_OS system, I get the following output:


$ nslookup pi.hole   
Server:		127.0.0.53
Address:	127.0.0.53#53

** server can't find pi.hole: NXDOMAIN

$ nslookup flurry.com
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	flurry.com
Address: 0.0.0.0
Name:	flurry.com
Address: ::

EDIT3: (Sorry...)

Since it may be relevant to the thread, here is the latest output from a cat /etc/resolv.conf on my Pop!_OS system and scutil --dns on my MacBook Pro both on the same network:

Pop!_OS:

$ cat /etc/resolv.conf 
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs must not access this file directly, but only through the
# symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
# replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad

MacBook Pro:

$ scutil --dns
DNS configuration

resolver #1
  nameserver[0] : 2400:4053:9421:500:2eff:65ff:fe44:4f11
  nameserver[1] : 192.168.0.69
  nameserver[2] : 192.168.0.69
  if_index : 5 (en0)
  flags    : Request A records, Request AAAA records
  reach    : 0x00000002 (Reachable)

resolver #2
  domain   : local
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300000

resolver #3
  domain   : 254.169.in-addr.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300200

resolver #4
  domain   : 8.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300400

resolver #5
  domain   : 9.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300600

resolver #6
  domain   : a.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300800

resolver #7
  domain   : b.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 301000

DNS configuration (for scoped queries)

resolver #1
  nameserver[0] : 2400:4053:9421:500:2eff:65ff:fe44:4f11
  nameserver[1] : 192.168.0.69
  nameserver[2] : 192.168.0.69
  if_index : 5 (en0)
  flags    : Scoped, Request A records, Request AAAA records
  reach    : 0x00000002 (Reachable)

Honestly, this is all pretty new to me and I assumed that I had to choose an IP address that was within that range.

UPDATE:

I checked back in on the pihole today and somehow the entry for interface wlan0 in /etc/dhcpcd.conf that I had deleted before running pihole -r yesterday (that seemed to have fixed somethings has returned and the IPv6 address of the pihole has somehow changed itself from ending in 1004 (which was set by the installation a mentioned and shown by hostname -I above on the RPI) to now ending in 1003. Now the pihole seems to be back to being bypassed on IPv6 as in the beginning of this thread. So confused... How would the pihole add an entry for static IP to the config file on its own?

What pihole -r did you run, reconfigure or repair?

IPv6 addresses change, that's how they work.

Thanks for the reply! I ran a reconfigure, not a repair.
Also, I deleted the interface wlan0 that had somehow reappeared and it seems to be back to where we were before blocking some ads but still being bypassed over IPv6. My TP-Link router does not seem to allow the setting of a custom IPv6 DNS server for LAN so I suppose I'm out of luck... Shutting off IPv6 would significantly reduce the quality/speed of my internet, right? I'd rather avoid that if possible.

Would shifting the DHCP server from the router to the pihole resolve the issue with IPv6 bypassing or is that completely unrelated?

No. Very few internet sites are IPv6 only. Given a choice of address, IPv6 may be slightly faster in some cases, but may be slower in others. Studies are not conclusive to show that IPv6 offers consistent speed improvements.

1 Like

I see. I was not aware of this! My ignorance led me to believe that IPv6 would be the superior option in all cases and turning it off would be trading off advantages in speed and congestion for the luxury of more complete ad-blocking. Thanks for the clarification!

If I opted to disable IPv6 on the router, would I then need to run another pihole -r to reconfigure the pihole for blocking only IPv4?

Switching off DHCPv6 (!) in your router would not be enough:

You'd have to consult your router's documentation sources on further details for its IPv6 configuration options.

If your router doesn't support configuring IPv6 DNS, you could consider disabling IPv6 altogether.

If you router doesn't support that either, your clients will bypass Pi-hole via IPv6.

Reconfigure will add a new block to the /etc/dhcpcd5.conf line if you set up a new IP address that isn't the same as the one that is currently there.

1 Like

IPv6 is great if you understand that it's not anything like IPv4 so all your knowledge of IPv4 does not apply. Trying to make IPv6 to behave the same as IPv4 will cause you days/weeks/months/years of headaches and surprise network failures. Unless you know you need IPv6 and you fully understand how IPv6 works (and does what ever it can to keep the network up, even if it has to completely ignore anything you told it to do since it's a protocol designed to self-heal and self-configure without any external management) then you're going to have a bad time.

That is exactly what happened it seems since I did change the IP of the pihole from 192.168.0.169 to 192.168.0.69 (to put it outside the 100-249 range of the router's DNS assignment as mentioned above) before performing said reconfigure! Thank you for the clarification.