Linux clients cannot reach update servers since activating Pi-Hole's DHCP server

Help
1

Hi all, first post. I've been using Pi-Hole successfully for a while now, despite not being any sort of a network expert. However, when reading the Query Log, I always found it difficult to ascertain which device was doing what, as the Client was always 'vodafone.broadband', and not the individual devices on the network. I figured out that this was happening because the router was handling DHCP, so I turned off the router's DHCP server, and activated the one in Pi-Hole.

This worked as I would have envisaged, in that I started to see the hostnames of individual devices in the Client column rather than that of the router. In fact, everything seemed to be working fine until one of my Linux Mint machines checked for updates, and the procedure hung. The same thing happened to the other one, and also to an Ubuntu laptop.

If I tether any of these three machines to my phone the update procedure completes successfully, so I know this must have been caused by something I've done here, but I'm stumped now. Any help would be gratefully received!

Token from pihole -d: sBhznsmJ

Thanks.

2

This only happens when you set Pi-hole as the WAN DNS server on the router, but your debug log shows your router is correctly advertising Pi-hole's IP as DNS server:

dns-server: 192.168.1.108

You should be able to see individual devices in the query log.

From one of your Linux devices, what is the output of these commands?

nslookup google.com
nslookup google.com 192.168.1.108
3

Yes, since I activated Pi-Hole's DHCP server I'm seeing individual devices in the log.

nslookup google.com

Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	google.com
Address: 142.250.200.14
Name:	google.com
Address: 2a00:1450:4009:822::200e

nslookup google.com 192.168.1.108

Server:		192.168.1.108
Address:	192.168.1.108#53

Non-authoritative answer:
Name:	google.com
Address: 142.250.200.14
Name:	google.com
Address: 2a00:1450:4009:822::200e

Thanks.

4

Oh, sorry... I read your debug log too fast and didn't notice the DHCP server is Pi-hole.

Where did you run the nslookup commands?
That machine is using itself (127.0.0.53) as DNS server, as the first command shows.

Did you renew the DHCP lease on the Linux machines, after setting Pi-hole as DHCP server? Maybe the Linux clients are still using the router lease.

5

I ran the nslookup commands in a terminal on one of the Linux clients.

I thought I'd renewed the leases, I used

sudo dhclient -r
sudo dhclient

on each of them. I've just done that again on the one I'm on now, and I still get the same results for the nslookup command. I've retraced my steps just to be sure, but I'm at a loss as to where it's getting 127.0.0.53 from?

6

Whenever you see a loopback IP (127.0.0.0/8) as a nameserver in the resolv.conf file, it means most likely a DNS stub resolver is active that does caching of queries made by local processes.
You can figure out which stub resolver with below one:

sudo ss -nltup sport = 53

Because your stub is @127.0.0.53, I suspect this to be systemd-resolved:

$ man systemd-resolved.service
[..]
       •   Additionally, systemd-resolved provides a local DNS stub
           listener on IP address 127.0.0.53 on the local loopback
           interface. Programs issuing DNS requests directly, bypassing
           any local API may be directed to this stub, in order to connect
           them to systemd-resolved. Note however that it is strongly
           recommended that local programs use the glibc NSS or bus APIs
           instead (as described above), as various network resolution
           concepts (such as link-local addressing, or LLMNR Unicode
           domains) cannot be mapped to the unicast DNS protocol.

       The DNS servers contacted are determined from the global settings
       in /etc/systemd/resolved.conf, the per-link static settings in
       /etc/systemd/network/*.network files (in case systemd-
       networkd.service(8) is used), the per-link dynamic settings
       received over DHCP, information provided via resolvectl(1), and any
       DNS server information made available by other system services. See
       resolved.conf(5) and systemd.network(5) for details about systemd's
       own configuration files for DNS servers. To improve compatibility,
       /etc/resolv.conf is read in order to discover configured system DNS
       servers, but only if it is not a symlink to
       /run/systemd/resolve/stub-resolv.conf, /usr/lib/systemd/resolv.conf
       or /run/systemd/resolve/resolv.conf (see below).

You can see how its configured with below one:

systemd-resolve --status

7 
sudo ss -nltup sport = 53

gives this:

Netid   State    Recv-Q   Send-Q     Local Address:Port     Peer Address:Port   Process                                                                         
udp     UNCONN   0        0             127.0.0.54:53            0.0.0.0:*       users:(("systemd-resolve",pid=791,fd=16))                                      
udp     UNCONN   0        0          127.0.0.53%lo:53            0.0.0.0:*       users:(("systemd-resolve",pid=791,fd=14))                                      
tcp     LISTEN   0        4096          127.0.0.54:53            0.0.0.0:*       users:(("systemd-resolve",pid=791,fd=17))                                      
tcp     LISTEN   0        4096       127.0.0.53%lo:53            0.0.0.0:*       users:(("systemd-resolve",pid=791,fd=15))

so it seems that you're right. I tried systemd-resolve --status but got only

systemd-resolve: command not found

but one of the config files I opened suggested resolvectl status which gives:

Global
         Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
  resolv.conf mode: stub

Link 2 (enp2s0f0)
    Current Scopes: none
         Protocols: -DefaultRoute -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 3 (wlp3s0)
    Current Scopes: DNS
         Protocols: +DefaultRoute -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.1.108
       DNS Servers: 192.168.1.108
        DNS Domain: lan

I had a look for the config files mentioned in the man page you posted, I don't have /etc/resolv.conf, but I found /run/systemd/resolve/stub-resolv.conf:

# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad
search lan

/usr/lib/systemd/resolv.conf:

# This file belongs to man:systemd-resolved(8). Do not edit.
#
# This is a static resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists no search
# domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs must not access this file directly, but only through the
# symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
# replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad
search .

and /run/systemd/resolve/stub-resolv.conf:

# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad
search lan

Is it just a case of editing these conf files and inserting the correct address (192.168.1.108)? Cheers.

8

That wouldn't be necessary, as your resolvectl output shows that your stub resolver is correctly using 192.168.1.108 for DNS already.

When those updates fail, you should inspect your Pi-hole's Query Log for blocked requests from a respective client.

9

Sorry, that must have been outdated or incorrect info.
I dont have any systems that run systemd-resolved to compare or try.
That systemd-resolve command is also not included in the systemd-resolved package.
But resolvectl is:

$ apt-file list systemd-resolved
systemd-resolved: /etc/systemd/resolved.conf
systemd-resolved: /lib/systemd/system/systemd-resolved.service
systemd-resolved: /lib/systemd/systemd-resolved
systemd-resolved: /sbin/resolvconf
systemd-resolved: /usr/bin/resolvectl
systemd-resolved: /usr/lib/sysusers.d/systemd-resolve.conf
systemd-resolved: /usr/lib/tmpfiles.d/systemd-resolve.conf
systemd-resolved: /usr/share/bash-completion/completions/resolvectl
systemd-resolved: /usr/share/bash-completion/completions/systemd-resolve
systemd-resolved: /usr/share/dbus-1/system-services/org.freedesktop.resolve1.service
systemd-resolved: /usr/share/dbus-1/system.d/org.freedesktop.resolve1.conf
systemd-resolved: /usr/share/doc/systemd-resolved/NEWS.Debian.gz
systemd-resolved: /usr/share/doc/systemd-resolved/changelog.Debian.gz
systemd-resolved: /usr/share/doc/systemd-resolved/copyright
systemd-resolved: /usr/share/lintian/overrides/systemd-resolved
systemd-resolved: /usr/share/man/man1/resolvconf.1.gz
systemd-resolved: /usr/share/man/man1/resolvectl.1.gz
systemd-resolved: /usr/share/man/man5/org.freedesktop.resolve1.5.gz
systemd-resolved: /usr/share/man/man5/resolved.conf.5.gz
systemd-resolved: /usr/share/man/man5/resolved.conf.d.5.gz
systemd-resolved: /usr/share/man/man8/systemd-resolved.8.gz
systemd-resolved: /usr/share/man/man8/systemd-resolved.service.8.gz
systemd-resolved: /usr/share/polkit-1/actions/org.freedesktop.resolve1.policy
systemd-resolved: /usr/share/zsh/vendor-completions/_resolvectl

As Bucking_Horn already mentioned, if 192.168.1.108 is Pi-hole, this client is already correctly configured.
But suppose you want to change the DNS servers manually for a desktop/workstation system, you shouldn't do that via systemd-resolved.
Most likely it runs separate software to configure systemd-resolved like for example network-manager:

$ apt show network-manager
[..]
Description: network management framework (daemon and userspace tools)
 NetworkManager is a system network service that manages your network devices
 and connections, attempting to keep active network connectivity when
 available. It manages ethernet, Wi-Fi, mobile broadband (WWAN), and PPPoE
 devices, and provides VPN integration with a variety of different VPN
 services.

$ man NetworkManager.conf
[..]
       dns
           Set the DNS processing mode.

           If the key is unspecified, default is used, unless
           /etc/resolv.conf is a symlink to
           /run/systemd/resolve/stub-resolv.conf,
           /run/systemd/resolve/resolv.conf, /lib/systemd/resolv.conf or
           /usr/lib/systemd/resolv.conf. In that case, systemd-resolved is
           chosen automatically.

           default: NetworkManager will update /etc/resolv.conf to reflect
           the nameservers provided by currently active connections. The
           rc-manager setting (below) controls how this is done.

           dnsmasq: NetworkManager will run dnsmasq as a local caching
           nameserver, using "Conditional Forwarding" if you are connected
           to a VPN, and then update resolv.conf to point to the local
           nameserver. It is possible to pass custom options to the
           dnsmasq instance by adding them to files in the
           "/etc/NetworkManager/dnsmasq.d/" directory. Note that when
           multiple upstream servers are available, dnsmasq will initially
           contact them in parallel and then use the fastest to respond,
           probing again other servers after some time. This behavior can
           be modified passing the 'all-servers' or 'strict-order' options
           to dnsmasq (see the manual page for more details).

           systemd-resolved: NetworkManager will push the DNS
           configuration to systemd-resolved

           unbound: NetworkManager will talk to unbound and
           dnssec-triggerd, using "Conditional Forwarding" with DNSSEC
           support.  /etc/resolv.conf will be managed by dnssec-trigger
           daemon.

           none: NetworkManager will not modify resolv.conf. This implies
           rc-manager unmanaged

           Note that the plugins dnsmasq, systemd-resolved and unbound are
           caching local nameservers. Hence, when NetworkManager writes
           /run/NetworkManager/resolv.conf and /etc/resolv.conf (according
           to rc-manager setting below), the name server there will be
           localhost only. NetworkManager also writes a file
           /run/NetworkManager/no-stub-resolv.conf that contains the
           original name servers pushed to the DNS plugin.

           When using dnsmasq and systemd-resolved per-connection added
           dns servers will always be queried using the device the
           connection has been activated on.

EDIT: Or by systemd-networkd:

$ man systemd-networkd
[..]
DESCRIPTION
       systemd-networkd is a system service that manages networks. It
       detects and configures network devices as they appear, as well as
       creating virtual network devices.

EDIT2: I believe Ubuntu runs it through netplan:

$ apt show netplan.io
[..]
Description: YAML network configuration abstraction for various backends
 netplan reads YAML network configuration files which are written
 by administrators, installers, cloud image instantiations, or other OS
 deployments. During early boot it then generates backend specific
 configuration files in /run to hand off control of devices to a particular
 networking daemon.
 .
 Currently supported backends are networkd and NetworkManager.

Oh and lets not forget connman :wink:

$ apt show connman
[..]
Description: Intel Connection Manager daemon
 The Linux Connection Manager project provides a daemon for managing
 Internet connections within embedded devices running the Linux
 operating system. The Connection Manager is designed to be slim and to
 use as few resources as possible. It is fully modular system that
 can be extended through plug-ins. The plug-in approach allows for
 easy adaption and modification for various use cases.
 .
 ConnMan provies IPv4 and IPv6 connectivity via:
  * ethernet
  * WiFi, using wpasupplicant
  * Cellular, using oFono
  * Bluetooth, using bluez
 .
 ConnMan implements DNS resolving and caching, DHCP clients for both IPv4 and
 IPv6, link-local IPv4 address handling and tethering (IP connection sharing)
 to clients via USB, ethernet, WiFi, cellular and Bluetooth.
 .
 This package contains the connman daemon and its plugins.
10

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.