Possible nameserver issues PiHole v5.18.3, on Ubuntu running Docker

ShadowofIntent · February 25, 2025, 4:14pm

I'm having an issue with a fresh install of Pihole on my Ubuntu 24.04.1 server with Docker. This was previously working until I wiped and reinstalled my server. At this point I am confused about if the system service should be enabled or disabled. If I disable it, the resolv.conf gets deleted along with the symlink. I also found this post suggesting to change the symlink. I have no experience in that so I am coming here before I do anything. If I enable it, the nameserver will also be replaced with 127.0.0.53 instead of 127.0.0.1 (which I believe is the correct call but need help verifying).

This all started because of this message:

Failed to deploy a stack: compose up operation failed: Error response from daemon: driver failed programming external connectivity on endpoint pihole (d6f27aaa4e5144ace4f49d0d7df7a4528c9521e26ba53fe57882d0e2a3c4ffb8): failed to bind host port for 0.0.0.0:53:172.18.0.2:53/tcp: address already in use

That is when my questions come into play about disabling systemd to free up port 53 for PiHole.

I use OpenWRT on my router, here is where I have the Pihole (Ubunutu server) defined:

All my adlists say "Adlist with ID was inaccessible during last gravity run." when I check the Pihole diagnosis log. The lists are all accessible and valid.
If I run a gravity update, I get the following two errors for each list:

  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available

When I try to create a debug log, I get this:

  * The debug log can be uploaded to tricorder.pi-hole.net for sharing with developers only.
[i] Debug script running in automated mode
    * Using curl for transmission.
    * curl failed, contact Pi-hole support for assistance.
    * Error message: curl: (6) Could not resolve host: tricorder.pi-hole.net

[✗] There was an error uploading your debug log.
   * Please try again or contact the Pi-hole team for assistance.
   * A local copy of the debug log can be found at: /var/log/pihole/pihole_debug.log

Docker-compose:

version: "3"
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:2024.07.0
    ports:
      - 53:53/tcp
      - 53:53/udp
      - 67:67/udp
      - 80:80/tcp
    environment:
      TZ: America/New_York
      WEBPASSWORD: 
    volumes:
      - /home/tom/dockervolumes/pihole/etc:/etc/pihole
      - /home/tom/dockervolumes/pihole/dnsmasq.d:/etc/dnsmasq.d
    cap_add:
      - NET_ADMIN
    restart: unless-stopped

Nslookup:

tom@ubuntu-server:~$ nslookup github.com
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	github.com
Address: 140.82.113.4

tom@ubuntu-server:~$ nslookup github.com 192.168.200.151
;; communications error to 192.168.200.151#53: connection refused
;; communications error to 192.168.200.151#53: connection refused
;; communications error to 192.168.200.151#53: connection refused
;; no servers could be reached

tom@ubuntu-server:~$

Edit: I was able to pull this log as well:

 [i] Target: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
  [i] Status: Pending...
  [✗] Status: Connection Refused
  [✗] List download failed: using previously cached list

  [✓] Parsed 127471 exact domains and 0 ABP-style domains (ignored 1 non-domain entries)
      Sample of non-domain entries:
        - "0.0.0.0"
  [i] Target: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/pro.plus.txt
  [i] Status: Pending...
  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available
  [i] Target: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/tif.txt
  [i] Status: Pending...
  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available
  [i] Target: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/dyndns.txt
  [i] Status: Pending...
  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available
  [i] Target: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/hoster.txt
  [i] Status: Pending...
  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available
  [i] Target: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/spam-tlds-adblock.txt
  [i] Status: Pending...
  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available
  [i] Target: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/native.amazon.txt
  [i] Status: Pending...
  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available
  [i] Target: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/native.apple.txt
  [i] Status: Pending...
  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available
  [i] Target: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/native.samsung.txt
  [i] Status: Pending...
  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available
  [i] Building tree...
  [✓] Building tree
  [i] Swapping databases...
  [✓] Swapping databases
  [✓] The old database remains available
  [i] Number of gravity domains: 127471 (127471 unique domains)
  [i] Number of exact blacklisted domains: 7
  [i] Number of regex blacklist filters: 1
  [i] Number of exact whitelisted domains: 208
  [i] Number of regex whitelist filters: 3
  [i] Cleaning up stray matter...
  [✓] Cleaning up stray matter
  [✓] FTL is listening on port 53
     [✓] UDP (IPv4)
     [✓] TCP (IPv4)
     [✓] UDP (IPv6)
     [✓] TCP (IPv6)
  [✓] Pi-hole blocking is enabled
fatal: unable to access 'https://github.com/pi-hole/pi-hole/': Could not resolve host: github.com
fatal: unable to access 'https://github.com/pi-hole/web/': Could not resolve host: github.com
fatal: unable to access 'https://github.com/pi-hole/FTL/': Could not resolve host: github.com
  Pi-hole version is v5.18.3 (Latest: N/A)
  web version is v5.21 (Latest: N/A)
  FTL version is v5.25.2 (Latest: N/A)
  Container tag is: 2024.07.0
Stopping pihole-FTL
Terminated
Stopping pihole-FTL
Terminated
Stopping pihole-FTL
Terminated
Stopping pihole-FTL
Terminated
Stopping pihole-FTL
Terminated
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service _postFTL: stopping
s6-rc: info: service _postFTL successfully stopped
s6-rc: info: service lighttpd: stopping
Stopping lighttpd
s6-rc: info: service lighttpd successfully stopped
s6-rc: info: service pihole-FTL: stopping
Stopping pihole-FTL
Terminated
s6-rc: info: service pihole-FTL successfully stopped
s6-rc: info: service _startup: stopping
s6-rc: info: service _startup successfully stopped
s6-rc: info: service _uid-gid-changer: stopping
s6-rc: info: service _uid-gid-changer successfully stopped
s6-rc: info: service cron: stopping
Stopping cron
s6-rc: info: service cron successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped

Ladrien · February 25, 2025, 8:15pm

(d6f27aaa4e5144ace4f49d0d7df7a4528c9521e26ba53fe57882d0e2a3c4ffb8): failed to bind host port for 0.0.0.0:53:172.18.0.2:53/tcp: address already in use

This indicates that there is a program already binding to port 53, such as a DNS server.
On your host machine, please run lsof -i :53

ShadowofIntent · February 25, 2025, 8:30pm

The output is blank:

tom@ubuntu-server:~$ lsof -i :53
tom@ubuntu-server:~$

Ladrien · February 25, 2025, 8:35pm

Since the host machine looks fine, let's turn our attention to the docker bridge network. Specifically:

Let's try to adjust the docker config to remove both 53 port mappings, and just put in 53:53 to specify all protocols.

If that does not work, you may have an issue with the docker DHCP server, or a ghost container still running and using the 172.18.0.2 IP address.

EDIT: My apologies. I forgot to include sudo in the previous command. Before changing docker settings, could you re-run the command with sudo? such as sudo lsof -i :53

ShadowofIntent · February 25, 2025, 8:49pm

Oh no problem at all! Here is the output of sudo lsof -i :53:

tom@ubuntu-server:~$ sudo lsof -i :53
[sudo] password for tom: 
COMMAND   PID            USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
systemd-r 911 systemd-resolve   14u  IPv4    642      0t0  UDP _localdnsstub:domain 
systemd-r 911 systemd-resolve   15u  IPv4    643      0t0  TCP _localdnsstub:domain (LISTEN)
systemd-r 911 systemd-resolve   16u  IPv4    644      0t0  UDP _localdnsproxy:domain 
systemd-r 911 systemd-resolve   17u  IPv4    645      0t0  TCP _localdnsproxy:domain (LISTEN)
tom@ubuntu-server:~$

Ladrien · February 25, 2025, 8:59pm

It does look like systemd-resolved is the culprit here. there are two ways to go about this:

disable the systemd-resolved service and try to configure your host's DNS resolution through another method that does not involve a loopback address. This may present issues in the future when doing distro version upgrades, such as 22.04 > 24.04
Set your docker container to use the docker bridge network, and give it a separate IP address from your host. This does complicate things, but will make your docker life significantly easier in the future if you are up for the challenge. Details are here: Networking | Docker Docs

I would personally recommend configuring a docker bridge.

ShadowofIntent · February 25, 2025, 9:41pm

Ahh OK. I use Portainer as well, I can try it on that since I am pretty new to Docker. And actually, as I look, there is a pihole bridge already created under networks:

nero355 · February 25, 2025, 9:51pm

Actually I have been running Pi-Hole v6 Beta and systemd-resolved side by side on a Debian 12.8 system booted as Live ISO environment from an USB Stick

And for Docker Containers it's always the best to use MACVLAN for your network

Ladrien · February 25, 2025, 9:53pm

Ok, perfect. It looks like you'll want to use the bridge -> pihole_default and map it like 172.17.0.2:172.18.0.2 where 172.17.0.2 is the actual ip you'd connect to the pihole with, and 172.18.0.2 is the internal pihole bridge.

If there are any options in the pihole container to change it from host networking to bridge networking, that would be the way. I am assuming that your LAN is 172.17.0.1/16?

Ladrien · February 25, 2025, 10:32pm

I gave the wrong advice, sorry. You'll want to create a new network and use either the IPVLAN or MACVLAN driver, and use the same subnet as your LAN. That will be the network that you use, with the mapping being 192.168.200.151:172.18.0.2

Information on which driver you'd like to use can be found here: Network drivers | Docker Docs

ShadowofIntent · February 25, 2025, 10:35pm

FYI I did find this - Docker DHCP and Network Modes - Pi-hole documentation it says not to use the out of box network bridge so I made my own. Easy enough and I named it simply "pihole"

Also on the docker-compose page I found this and enabled it in my docker-compose (I can re-move if I'm misterpreting what it means):

      # If using Docker's default `bridge` network setting the dns listening mode should be set to 'all'
      FTLCONF_dns_listeningMode: 'all'

One thing I am confused about is how to map/define it in docker-compose. Is this how I would define it in the docker-compose?

   networks:
      - pihole  # Attach container to the existing bridge network

I am assuming that your LAN is 172.17.0.1/16?

Correct.

ShadowofIntent · February 25, 2025, 10:36pm

OK thanks, I'll poke around and get back to you.

Bucking_Horn · February 25, 2025, 10:41pm

Consider passing a public DNS server into your container:

services:
    pihole:
        ports:
        (…)
        environment:
        (…)
        volumes:
        (…)
        dns:
            - 127.0.0.1
            - 1.1.1.1
        (…)

That may fix gravity updates and should allow creation of a debug log.

Pi-hole definitely needs port 53, so any other service claiming it needs to be stopped, unless you'd run your Pi-hole container in macvlan or ipvlan network mode.

Ladrien · February 25, 2025, 10:51pm

I'd keep this for now, but since you are creating your own bridge, this shouldn't need to be all. Maybe consider changing this afterwards.

It would also be ideal to set the IP address of the container. You may have to edit it in the actual network itself. See: Networking | Docker Docs

ShadowofIntent · February 25, 2025, 11:07pm

I'm taking a shot at creating a macvlan since I've never done it. I'll report back.

ShadowofIntent · February 25, 2025, 11:30pm

This is the part I am stuck on for the moment:

Should I be using 192 addresses or 172? Judging by what you said earlier and the other networks listed they should be 172 addresses. I think this is where a little lack of knowledge is coming into play.

Ladrien · February 25, 2025, 11:34pm

I was basing the advice on a guess of your LAN subnet. If your LAN is indeed 172.17.0.1/16, then the docker network should match your LAN.
subnet: 172.17.0.1/16
gateway: 172.17.0.1
ip range: 172.17.0.1/16 You can probably lower the amount of IPs here, similar to what it's recommended. eg: 172.18.0.1/24. I'd leave it for now if you are unsure.

Bucking_Horn · February 25, 2025, 11:39pm

Since you want your macvlan container to join your home network, it would need an IP from your home network's subnet.

You'd have to check that, but I guess that likely would be some 192.68.x.0/24, as Docker would usually grab a narrower range from 172.16.0.0/12 for its internal networks.

ShadowofIntent · February 26, 2025, 12:04am

Ahh OK thank you for clearing that up. I was able to create the macvlan, I called it "pihole". Now I need to define it in docker-compose, I think this is correct. Thoughts?

---
version: "3"
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:2024.07.0
    ports:
      - 53:53/tcp
      - 53:53/udp
      - 67:67/udp
      - 80:80/tcp
    environment:
      TZ: America/New_York
      WEBPASSWORD:
    volumes:
      - /home/tom/dockervolumes/pihole/etc:/etc/pihole
      - /home/tom/dockervolumes/pihole/dnsmasq.d:/etc/dnsmasq.d
    cap_add:
      - NET_ADMIN
    networks:
      - pihole
    restart: unless-stopped

networks:
  pihole:
    external: true

Ladrien · February 26, 2025, 12:37am

I think portainer has a way to accomplish what you want through the webUI.

Regardless, I believe this should work, provided you do not need pi-hole to handle DHCP.

---
version: "3"
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:2024.07.0
    ports:
      - 53:53/tcp
      - 53:53/udp
     # - 67:67/udp
      - 80:80/tcp
    environment:
      TZ: America/New_York
      WEBPASSWORD: ******
    volumes:
      - /home/tom/dockervolumes/pihole/etc:/etc/pihole
      - /home/tom/dockervolumes/pihole/dnsmasq.d:/etc/dnsmasq.d
    #cap_add:
     # - NET_ADMIN
    networks:
      pihole:
        ipv4_address: 172.17.0.2
    restart: unless-stopped

networks:
  pihole:
    external: true