DNS not resolving between Client and Pihole

Hi Everyone,
I had a power outage for an extended period of time 4+ hrs. Once power was restored, I noticed internet was not working. After quick troubleshooting I knew none of my clients could resolve DNS.

Configuration

  • I have a Unifi network setup (UDMP, USW-16, etc) with multiple VLANs.
  • I have two pihole servers, 10.19.91.3 and 10.19.93.3 in different VLANs.
  • Every VLAN is assigned DNS via the network's DHCP service settings pointed at both Pihole servers.
  • My UDMP's internet DNS is set to auto.
  • I have FW rules in place to block alternate DNS requests outside of my list of Pihole Servers.

Troubleshooting

  • First, here is my pihole debug logs.
  • From my router and upstream ISP, DNS works fine. I can connect to Unifi's site manager, unify.ui.com.
  • From a client machine, if I run nslookup I see
   ❯ nslookup google.com           
   ;; Got SERVFAIL reply from 10.19.91.3, trying next server
   Server:		10.19.93.3
   Address:	10.19.93.3#53

   ** server can't find google.com: SERVFAIL

   ❯ nslookup pi.hole              
   Server:		10.19.91.3
   Address:	10.19.91.3#53

   Name:	pi.hole
   Address: 0.0.0.0
                                                                                                                                                                 
   ❯ nslookup flurry.com           
   Server:		10.19.91.3
   Address:	10.19.91.3#53

   Name:	flurry.com
   Address: 0.0.0.0
  • I intermittently switch the client's DNS to 1.1.1.1 and get successful resolution
   ❯ nslookup google.com           
   Server:		1.1.1.1
   Address:	1.1.1.1#53

   Non-authoritative answer:
   Name:	google.com
   Address: 108.177.122.101
   Name:	google.com
   Address: 108.177.122.100
   Name:	google.com
   Address: 108.177.122.138
   Name:	google.com
   Address: 108.177.122.102
   Name:	google.com
   Address: 108.177.122.113
   Name:	google.com
   Address: 108.177.122.139
  • Then, I SSH to the pihole host, 10.19.91.3, and performed a nslookup
   franklin@mgmt-pi:~ $ nslookup google.com
   Server:		127.0.0.1
   Address:	127.0.0.1#53

   ** server can't find google.com: SERVFAIL

Help

  1. Why can't the pihole server resolve DNS? Should my /etc/dhcpcd.conf be set to something different than static domain_name_servers=127.0.0.1?
  2. Viewing the first nslookup I show, you can see it attempted my first pihole server, 10.19.91.3, then tried the second one, 10.19.93.3. Seems like both servers were not working which makes me think I have a configuration error?

Check the date/time on the Pi-hole host. If it lost track of time (there is no onboard clock on most Pi's), DNS resultion may be lost. Accurate time is required to do DNSSEC validation, and you have DNSSEC enabled.