Single dot domain/DNS root zone query issue with containerized wireguard

deHakkelaar · September 20, 2022, 6:04pm

One idea sprung to mind, maybe you can localise more closely where the truncating is happening in the chain.
You just have to query a DNS server for a record that is local to the DNS server (doenst need to be forwarded) and see if it gets truncated.

For me, my router is @10.0.0.1:

pi@ph5b:~ $ dig +notcp @10.0.0.1 -x 10.0.0.1
[..]
router.asus.com.

And Pi-hole is @10.0.0.4:

pi@ph5b:~ $ dig +notcp @10.0.0.4 -x 10.0.0.4
[..]
pi.hole.

My ISP DNS servers:

pi@ph5b:~ $ dig +notcp @195.121.1.34 -x 195.121.1.34
[..]
ns1.wxs.nl.

Cloudflare:

pi@ph5b:~ $ dig +notcp @1.1.1.1 -x 1.1.1.1
[..]
one.one.one.one.

Not sure though and maybe another type of query instead of PTR is better suited bc maybe PTR queries dont get truncated like the NS one does

armujahid · October 1, 2022, 5:32pm

For all 4 cases dig command is working fine. only nslookup with . is being truncated. I have also run those nslookups in some AWS VM that is another country and of course isn't using pihole. Note that nslooup of dot domain is still being truncated with cloudflare

ubuntu@someAWSinstance:~$ nslookup -type=NS . 1.1.1.1
;; Truncated, retrying in TCP mode.
Server:		1.1.1.1
Address:	1.1.1.1#53

.	nameserver = a.root-servers.net.
.	nameserver = b.root-servers.net.
.	nameserver = c.root-servers.net.
.	nameserver = d.root-servers.net.
.	nameserver = e.root-servers.net.
.	nameserver = f.root-servers.net.
.	nameserver = g.root-servers.net.
.	nameserver = h.root-servers.net.
.	nameserver = i.root-servers.net.
.	nameserver = j.root-servers.net.
.	nameserver = k.root-servers.net.
.	nameserver = l.root-servers.net.
.	nameserver = m.root-servers.net.

ubuntu@someAWSinstance:~$ nslookup -type=NS .
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
.	nameserver = a.root-servers.net.
.	nameserver = d.root-servers.net.
.	nameserver = c.root-servers.net.
.	nameserver = b.root-servers.net.
.	nameserver = j.root-servers.net.
.	nameserver = k.root-servers.net.
.	nameserver = g.root-servers.net.
.	nameserver = m.root-servers.net.
.	nameserver = f.root-servers.net.
.	nameserver = e.root-servers.net.
.	nameserver = h.root-servers.net.
.	nameserver = l.root-servers.net.
.	nameserver = i.root-servers.net.

Authoritative answers can be found from:

ubuntu@someAWSinstance:~$ nslookup -type=NS . 8.8.8.8
Server:		8.8.8.8
Address:	8.8.8.8#53

Non-authoritative answer:
.	nameserver = a.root-servers.net.
.	nameserver = b.root-servers.net.
.	nameserver = c.root-servers.net.
.	nameserver = d.root-servers.net.
.	nameserver = e.root-servers.net.
.	nameserver = f.root-servers.net.
.	nameserver = g.root-servers.net.
.	nameserver = h.root-servers.net.
.	nameserver = i.root-servers.net.
.	nameserver = j.root-servers.net.
.	nameserver = k.root-servers.net.
.	nameserver = l.root-servers.net.
.	nameserver = m.root-servers.net.

Authoritative answers can be found from:

You can also try nslookup -type=NS . 1.1.1.1 and it should truncate. So issue is specific to cloudflare's DNS

deHakkelaar · October 2, 2022, 3:16am

Yes I get the same results as you with nslookup.
But am not sure what nslookup does under the hood.

Where 1.1.1.1 is different compared to the others is that it supplies ADDITIONAL records:

pi@ph5b:~ $ dig +notcp @1.1.1.1 . ns

; <<>> DiG 9.16.22-Raspbian <<>> +notcp +additional @1.1.1.1 . ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7181
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 27

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;.                              IN      NS

;; ANSWER SECTION:
.                       511735  IN      NS      a.root-servers.net.
.                       511735  IN      NS      b.root-servers.net.
.                       511735  IN      NS      c.root-servers.net.
.                       511735  IN      NS      d.root-servers.net.
.                       511735  IN      NS      e.root-servers.net.
.                       511735  IN      NS      f.root-servers.net.
.                       511735  IN      NS      g.root-servers.net.
.                       511735  IN      NS      h.root-servers.net.
.                       511735  IN      NS      i.root-servers.net.
.                       511735  IN      NS      j.root-servers.net.
.                       511735  IN      NS      k.root-servers.net.
.                       511735  IN      NS      l.root-servers.net.
.                       511735  IN      NS      m.root-servers.net.

;; ADDITIONAL SECTION:
a.root-servers.net.     511735  IN      A       198.41.0.4
a.root-servers.net.     511735  IN      AAAA    2001:503:ba3e::2:30
b.root-servers.net.     511735  IN      A       199.9.14.201
b.root-servers.net.     511735  IN      AAAA    2001:500:200::b
c.root-servers.net.     511735  IN      A       192.33.4.12
c.root-servers.net.     511735  IN      AAAA    2001:500:2::c
d.root-servers.net.     511735  IN      A       199.7.91.13
d.root-servers.net.     511735  IN      AAAA    2001:500:2d::d
e.root-servers.net.     511735  IN      A       192.203.230.10
e.root-servers.net.     511735  IN      AAAA    2001:500:a8::e
f.root-servers.net.     511735  IN      A       192.5.5.241
f.root-servers.net.     511735  IN      AAAA    2001:500:2f::f
g.root-servers.net.     511735  IN      A       192.112.36.4
g.root-servers.net.     511735  IN      AAAA    2001:500:12::d0d
h.root-servers.net.     511735  IN      A       198.97.190.53
h.root-servers.net.     511735  IN      AAAA    2001:500:1::53
i.root-servers.net.     511735  IN      A       192.36.148.17
i.root-servers.net.     511735  IN      AAAA    2001:7fe::53
j.root-servers.net.     511735  IN      A       192.58.128.30
j.root-servers.net.     511735  IN      AAAA    2001:503:c27::2:30
k.root-servers.net.     511735  IN      A       193.0.14.129
k.root-servers.net.     511735  IN      AAAA    2001:7fd::1
l.root-servers.net.     511735  IN      A       199.7.83.42
l.root-servers.net.     511735  IN      AAAA    2001:500:9f::42
m.root-servers.net.     511735  IN      A       202.12.27.33
m.root-servers.net.     511735  IN      AAAA    2001:dc3::35

;; Query time: 19 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Sun Oct 02 03:31:06 CEST 2022
;; MSG SIZE  rcvd: 811

Vs Google @8.8.8.8:

pi@ph5b:~ $ dig +notcp @8.8.8.8 . ns

; <<>> DiG 9.16.22-Raspbian <<>> +notcp +additional @8.8.8.8 . ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54956
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;.                              IN      NS

;; ANSWER SECTION:
.                       64927   IN      NS      a.root-servers.net.
.                       64927   IN      NS      b.root-servers.net.
.                       64927   IN      NS      c.root-servers.net.
.                       64927   IN      NS      d.root-servers.net.
.                       64927   IN      NS      e.root-servers.net.
.                       64927   IN      NS      f.root-servers.net.
.                       64927   IN      NS      g.root-servers.net.
.                       64927   IN      NS      h.root-servers.net.
.                       64927   IN      NS      i.root-servers.net.
.                       64927   IN      NS      j.root-servers.net.
.                       64927   IN      NS      k.root-servers.net.
.                       64927   IN      NS      l.root-servers.net.
.                       64927   IN      NS      m.root-servers.net.

;; Query time: 9 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Oct 02 03:41:52 CEST 2022
;; MSG SIZE  rcvd: 239

Quad9 (9.9.9.9) and Level3 (4.2.2.1) reply the same as Google.

And I can reproduce the truncating with dig if I tell it to drop EDNS support (+noedns) which limits UDP packet size to 512 bytes max:

pi@ph5b:~ $ dig +notcp +noedns @1.1.1.1 . ns
;; Truncated, retrying in TCP mode.

; <<>> DiG 9.16.22-Raspbian <<>> +notcp +additional +noedns @1.1.1.1 . ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5891
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 26

;; QUESTION SECTION:
;.                              IN      NS

;; ANSWER SECTION:
.                       510468  IN      NS      a.root-servers.net.
.                       510468  IN      NS      b.root-servers.net.
.                       510468  IN      NS      c.root-servers.net.
.                       510468  IN      NS      d.root-servers.net.
.                       510468  IN      NS      e.root-servers.net.
.                       510468  IN      NS      f.root-servers.net.
.                       510468  IN      NS      g.root-servers.net.
.                       510468  IN      NS      h.root-servers.net.
.                       510468  IN      NS      i.root-servers.net.
.                       510468  IN      NS      j.root-servers.net.
.                       510468  IN      NS      k.root-servers.net.
.                       510468  IN      NS      l.root-servers.net.
.                       510468  IN      NS      m.root-servers.net.

;; ADDITIONAL SECTION:
a.root-servers.net.     510468  IN      A       198.41.0.4
a.root-servers.net.     510468  IN      AAAA    2001:503:ba3e::2:30
b.root-servers.net.     510468  IN      A       199.9.14.201
b.root-servers.net.     510468  IN      AAAA    2001:500:200::b
c.root-servers.net.     510468  IN      A       192.33.4.12
c.root-servers.net.     510468  IN      AAAA    2001:500:2::c
d.root-servers.net.     510468  IN      A       199.7.91.13
d.root-servers.net.     510468  IN      AAAA    2001:500:2d::d
e.root-servers.net.     510468  IN      A       192.203.230.10
e.root-servers.net.     510468  IN      AAAA    2001:500:a8::e
f.root-servers.net.     510468  IN      A       192.5.5.241
f.root-servers.net.     510468  IN      AAAA    2001:500:2f::f
g.root-servers.net.     510468  IN      A       192.112.36.4
g.root-servers.net.     510468  IN      AAAA    2001:500:12::d0d
h.root-servers.net.     510468  IN      A       198.97.190.53
h.root-servers.net.     510468  IN      AAAA    2001:500:1::53
i.root-servers.net.     510468  IN      A       192.36.148.17
i.root-servers.net.     510468  IN      AAAA    2001:7fe::53
j.root-servers.net.     510468  IN      A       192.58.128.30
j.root-servers.net.     510468  IN      AAAA    2001:503:c27::2:30
k.root-servers.net.     510468  IN      A       193.0.14.129
k.root-servers.net.     510468  IN      AAAA    2001:7fd::1
l.root-servers.net.     510468  IN      A       199.7.83.42
l.root-servers.net.     510468  IN      AAAA    2001:500:9f::42
m.root-servers.net.     510468  IN      A       202.12.27.33
m.root-servers.net.     510468  IN      AAAA    2001:dc3::35

;; Query time: 9 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Sun Oct 02 03:47:28 CEST 2022
;; MSG SIZE  rcvd: 800

As the others dont supply ADDITIONAL records, the 239 bytes answer they return is well below the 512 bytes UDP limit.
And the 800 bytes answer from 1.1.1.1, with ADDITIONAL records, exceeds the 512 bytes limit (if no EDNS support).
Thats the only thing I can think of causing the truncating when querying 1.1.1.1 if compare with the others.
Something along the path upstream to 1.1.1.1 limiting UDP packet size or doesnt support EDNS which amongst others allows larger packets.
And if I did my research correctly, ADDITIONAL records only comes with EDNS (correct me if I'm wrong pls?):

https://www.ietf.org/rfc/rfc2671.txt

I also noticed an inconsistency in number of ADDITIONAL records when flipping +noedns which I cant explain:

pi@ph5b:~ $ dig +notcp @1.1.1.1 . ns
[..]
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 27

pi@ph5b:~ $ dig +notcp +noedns @1.1.1.1 . ns
[..]
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 26

I counted them for both ouputs and it should be 26.

Am not sure if all this is related with the original issue from your OP though
Have you tried selecting another upstream DNS provider, not Cloudflare, to see if the excessive dot queries stop?

armujahid · October 2, 2022, 8:17am

Excessive dot queues aren't stopping even if I directly select google dns from pihole. But at least reply is now different (BLOB vs NODATA that I was getting from cloudflare).

Oct  2 13:13:28 dnsmasq[3301]: query[NS] . from 172.18.0.1
Oct  2 13:13:28 dnsmasq[3301]: forwarded . to 8.8.8.8
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:28 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: query[NS] . from 172.18.0.1
Oct  2 13:13:29 dnsmasq[3301]: forwarded . to 8.8.8.8
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>
Oct  2 13:13:29 dnsmasq[3301]: reply . is <NS>

deHakkelaar · October 2, 2022, 9:07am

Thats at least a small improvement

I dont know how to troubleshoot that Wireguard container though as I have little experience with Docker.
But if you post some more info like:
What HW?
Did you follow a guide?
Which container(source/link)?
How is the container created (Docker compose, YAML, Portainer etc)?
Docker networking mode (host, macvlan, bridge)?

Maybe someone else might chip in?
And oc you could also try to get help via the Wireguard container support channels.

armujahid · October 2, 2022, 9:34am

Thanks. Yeah I will also post this in wireguard forum.
pihole docker-compse:

Old wireguard config that I was using while reporting this issue:

version: "2.1"
services:
  wireguard:
    image: ghcr.io/linuxserver/wireguard
    container_name: wireguard
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Karachi
      - SERVERPORT=51820 #optional
      - PEERS=something #optional
      - PEERDNS=auto #optional
    volumes:
      - ./config:/config
      - ./libmodules:/lib/modules
    ports:
      - 51820:51820/udp
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: unless-stopped

Newer Wireguard docker-compose that isn't even using host DNS (PEERDNS is not auto):

version: "2.1"
services:
  wireguard:
    image: lscr.io/linuxserver/wireguard:latest
    container_name: wireguard
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Karachi
      - SERVERPORT=51820 #optional
      - PEERS=Mi11Tpro #optional
      - PEERDNS=8.8.8.8 #optional
      # - PEERDNS=auto #optional
      # - INTERNAL_SUBNET=10.13.13.0 #optional
      - ALLOWEDIPS=0.0.0.0/0 #optional
      - LOG_CONFS=true #optional
    volumes:
      - ./config:/config
      - ./libmodules:/lib/modules
    ports:
      - 51820:51820/udp
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: unless-stopped

HW: raspberry pi 4b 4GB
OS:

Linux raspberrypi 5.15.32-v7l+ #1538 SMP Thu Mar 31 19:39:41 BST 2022 armv7l GNU/Linux
Distributor ID:	Raspbian
Description:	Raspbian GNU/Linux 11 (bullseye)
Release:	11
Codename:	bullseye

armujahid · October 2, 2022, 9:53am

I have also linked this thread in wireguard reddit

armujahid · December 25, 2022, 5:54pm

I got it fixed by binding the DNS listening port to the pihole node local address in the docker-compose file of Pi-hole,
in my case - "192.168.50.11:53:53/udp" instead of - "53:53/udp".

I am not sure why this solution worked. Solution was linked by someone on reddit. I have also added this in my reddit post liked above.
Related links:

github.com/linuxserver/docker-wireguard

Wireguard is performing an invalid DNS lookup about once per second

opened 04:21AM - 04 Mar 21 UTC

closed 03:18PM - 03 May 22 UTC

texneus

no-issue-activity

[linuxserverurl]: https://linuxserver.io [![linuxserver.io](https://raw.githubu…sercontent.com/linuxserver/docker-templates/master/linuxserver.io/img/linuxserver_medium.png)][linuxserverurl] ------------------------------ ## Expected Behavior DNS lookups should only occur for legitimate web addresses. Lookups should stop or time out when no response is received rather than continue indefinately. ## Current Behavior My pihole logs are showing repeated (about 1x per second) requests from client IP 172.17.0.1 (the docker0 network adapter) to lookup ".". This has been going on since I set the container up (going on 2-3 weeks now). The activity stops when the Wireguard docker container is stopped. I have set pihole to block the request as otherwise pihole forwards the request to it's DNS provider. Blocking the DNS does not appear to affect wireguard functionality. There are no other Docker containers or services on this system (beyond those enabled by default with Raspian). ## Steps to Reproduce 1. Install pihole/pihole docker container and setup 2. Install linuxerver/wireguard docker container and setup (on the same system with pihole/pihole) 3. Observe repeated DNS lookups from 172.17.0.1 to "." in the pihole logs: ![Screenshot_20210303_214743](https://user-images.githubusercontent.com/23441049/109908694-5470fa80-7c6a-11eb-9463-1061823490ce.jpeg) ## Environment **OS:** Raspian 10 **CPU architecture:** arm32 (Raspberry Pi 4) **How docker service was installed:** From the official docker repo ## Command used to create docker container (run/create/compose/screenshot) ``` # docker create \ --name=wireguard \ --cap-add=NET_ADMIN \ --cap-add=SYS_MODULE \ --restart=on-failure:5 \ -e PUID=1000 \ -e PGID=1000 \ -e TZ="America/Chicago" \ -e SERVERURL=xxxxxxx.yyyyyyyy.zzz \ -e SERVERPORT=51820 \ -e PEERS=MPhone,MLaptop \ -e PEERDNS=auto \ -e INTERNAL_SUBNET=10.13.13.0 \ -p 51820:51820/udp \ -v /opt/wireguard/data/config:/config \ -v /lib/modules:/lib/modules \ -v /usr/src:/usr/src \ linuxserver/wireguard # docker start wireguard ``` ## Docker logs ``` # docker logs wireguard [s6-init] making user provided files available at /var/run/s6/etc...exited 0. [s6-init] ensuring user provided files have correct perms...exited 0. [fix-attrs.d] applying ownership & permissions fixes... [fix-attrs.d] done. [cont-init.d] executing container initialization scripts... [cont-init.d] 01-envfile: executing... [cont-init.d] 01-envfile: exited 0. [cont-init.d] 10-adduser: executing... ------------------------------------- _ () | | ___ _ __ | | / __| | | / \ | | \__ \ | | | () | |_| |___/ |_| \__/ Brought to you by linuxserver.io ------------------------------------- To support the app dev(s) visit: WireGuard: https://www.wireguard.com/donations/ To support LSIO projects visit: https://www.linuxserver.io/donate/ ------------------------------------- GID/UID ------------------------------------- User uid: 1000 User gid: 1000 ------------------------------------- [cont-init.d] 10-adduser: exited 0. [cont-init.d] 30-config: executing... Uname info: Linux fd7d47c206fe 5.10.11-v7l+ #1399 SMP Thu Jan 28 12:09:48 GMT 2021 armv7l armv7l armv7l GNU/Linux **** It seems the wireguard module is already active. Skipping kernel header install and module compilation. **** **** Server mode is selected **** **** External server address is set to xxxxxxx.yyyyyyyy.zzz **** **** External server port is set to 51820. Make sure that port is properly forwarded to port 51820 inside this container **** **** Internal subnet is set to 10.13.13.0 **** **** AllowedIPs for peers 0.0.0.0/0, ::/0 **** **** PEERDNS var is either not set or is set to "auto", setting peer DNS to 10.13.13.1 to use wireguard docker host's DNS. **** **** Server mode is selected **** **** No changes to parameters. Existing configs are used. **** [cont-init.d] 30-config: exited 0. [cont-init.d] 99-custom-scripts: executing... [custom-init] no custom files found exiting... [cont-init.d] 99-custom-scripts: exited 0. [cont-init.d] done. [services.d] starting services [services.d] done. [#] ip link add wg0 type wireguard .:53 CoreDNS-1.8.3 linux/arm, go1.16, 4293992 [#] wg setconf wg0 /dev/fd/63 [#] ip -4 address add 10.13.13.1 dev wg0 [#] ip link set mtu 1420 up dev wg0 [#] ip -4 route add 10.13.13.3/32 dev wg0 [#] ip -4 route add 10.13.13.2/32 dev wg0 [#] iptables -A FORWARD -i wg0 -j ACCEPT; iptables -A FORWARD -o wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE ```

Note that a user in docker-wireguard github issue is blaming pihole docker for this.

armujahid · March 1, 2023, 8:27am

I reverted the solution because after making the change, the Pi-hole Docker container failed to restart automatically after a system reboot.

At the moment, I have stopped using the WireGuard container, and everything appears to be functioning correctly.