Pihole not listening on wireguard interface despite "Listen on all interfaces" is set

I have a working Pihole configuration and added wireguard to my device. The vpn connection workes (can ping my pihole device) but Pihole doesn't listen on the wireguard interface. If selected " Listen on all interfaces" (first option in web gui) I canconfirm that /etc/dnsmasq.d/01-pihole.conf contains the line local-service.

The wireguard interface is established

nanopi@nanopineoplus2:~$ sudo ifconfig 
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.178.70  netmask 255.255.255.0  broadcast 192.168.178.255
        inet6 fe80::c897:3067:783a:44c1  prefixlen 64  scopeid 0x20<link>
        ether 02:01:3e:5b:68:08  txqueuelen 1000  (Ethernet)
        RX packets 146267  bytes 18310575 (17.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 108158  bytes 41761823 (39.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 25  

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 16352  bytes 1568490 (1.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 16352  bytes 1568490 (1.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wg0: flags=209<UP,POINTOPOINT,RUNNING,NOARP>  mtu 1420
        inet 10.0.40.6  netmask 255.255.255.255  destination 10.0.40.6
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 1000  (UNSPEC)
        RX packets 60698  bytes 6208510 (5.9 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 80195  bytes 36038144 (34.3 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 8c:f7:10:a0:a8:99  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Restarting pihole-FTL doesn't solve it. Full restart of the whole system also not. System is configured that pihole-ftl.service waits for wg-quick@wg0.service to be up.

Switching to " Listen on all interfaces, permit all origins" (3. option GUI) makes it work immediately. In /etc/dnsmasq.d/01-pihole.conf the line local-service is exchanged for except-interface=nonexisting

Any ideas why it is not working with " Listen on all interfaces" ?


nanopi@nanopineoplus2:~$ pihole -v
  Pi-hole version is v4.3.5-458-g2de5362 (Latest: v4.4)
  AdminLTE version is v4.3.2-457-g0a81dadf (Latest: v4.3.3)
  FTL version is vDev-78e0332 (Latest: v4.3.

https://tricorder.pi-hole.net/png745ugfg

I digged into this "Listen on interfaces" feature a while ago: dnsmasq fails to identify LAN requests when started before network ¡ Issue #2924 ¡ pi-hole/pi-hole ¡ GitHub
It fails to identify "local interfaces" if the network has not been fully configured before pihole-FTL starts. But this is obviously not your issue. I am not sure how dnsmasq identifies "local interfaces", for WireGuard however it worked during our test: DietPi-Software | WireGuard + Pi-hole ¡ Issue #3007 ¡ MichaIng/DietPi ¡ GitHub

Thanks for your answer. I found those issues already but no information to solve my present problem.
The strange thing is - it has already worked. But somehow (pihole update? package update?) it's not working anymore.

OS: Armbian (Debian Buster)

I think this is the core question here. Which requirements need to be fulfilled before dnsmasq accepts an interface as local. Is there a way to list all interfaces dnsmasq/pihole-ftl listens on?

nanopi@nanopineoplus2:~$ sudo netstat -tulnp|grep -i list
tcp        0      0 0.0.0.0:53              0.0.0.0:*               LISTEN      1489/pihole-FTL     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      970/sshd            
tcp        0      0 127.0.0.1:4711          0.0.0.0:*               LISTEN      1489/pihole-FTL     
tcp        0      0 127.0.0.1:5353          0.0.0.0:*               LISTEN      1037/unbound        
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1418/lighttpd       
tcp6       0      0 :::53                   :::*                    LISTEN      1489/pihole-FTL     
tcp6       0      0 :::22                   :::*                    LISTEN      970/sshd            
tcp6       0      0 ::1:4711                :::*                    LISTEN      1489/pihole-FTL     
tcp6       0      0 ::1:5353                :::*                    LISTEN      1037/unbound        
tcp6       0      0 :::80                   :::*                    LISTEN      1418/lighttpd   
nanopi@nanopineoplus2:~$ sudo ss -lpn |grep pihole*
u_str LISTEN 0      5                      /var/run/pihole/FTL.sock 12507                                                 * 0                                    users:(("pihole-FTL",pid=1489,fd=12))                                          
udp   UNCONN 0      0                                       0.0.0.0:53                                              0.0.0.0:*                                    users:(("pihole-FTL",pid=1489,fd=4))                                           
udp   UNCONN 0      0                                          [::]:53                                                 [::]:*                                    users:(("pihole-FTL",pid=1489,fd=6))                                           
tcp   LISTEN 0      32                                      0.0.0.0:53                                              0.0.0.0:*                                    users:(("pihole-FTL",pid=1489,fd=5))                                           
tcp   LISTEN 0      5                                     127.0.0.1:4711                                            0.0.0.0:*                                    users:(("pihole-FTL",pid=1489,fd=10))                                          
tcp   LISTEN 0      32                                         [::]:53                                                 [::]:*                                    users:(("pihole-FTL",pid=1489,fd=7))                                           
tcp   LISTEN 0      5                                         [::1]:4711                                               [::]:*                                    users:(("pihole-FTL",pid=1489,fd=11))  

Since dnsmasq fails to identify local interfaces/clients when interfaces have not been fully configured before, it seems to derive this info from present configured interfaces. I could image that it checks assigned IPs of configured interfaces (like ip a) and their netmasks and identifies a client as local if it matches one of those subnets. Not 100% sure if WireGuard (wg0) is identified always correctly.

Hmm now that I recheck your ifconfig for wg0:

inet 10.0.40.6 netmask 255.255.255.255

This would mean that the wg0 subnet consists of 10.0.40.6 (the server) only.

Compare with a proper eth0 entry that includes the whole IP range into the subnet:

inet 192.168.178.70 netmask 255.255.255.0

The .0 in netmask defines the subnet to contain all 192.168.178.* IPs instead of 192.168.178.70 only.

Can you please paste your wg0.conf (removing all key strings etc of course)?

Checking the dnsmasq code, it seems it scans the currently available interfaces and their configurations for each individual query.

1 Like

If pihole-FTL/dnsmasq is indeed ignoring some of your subnets, you should see corresponding errors like "Ignoring query from non-local network" in your logs:

grep -ni ignoring /var/log/pihole-FTL.log

If you don't, then I would suspect a different cause instead, e.g. routing / firewall issues.

With regards to the service configurations used by you as linked above:
By intuition, wouldn't Wireguard be loosely dependent on DNS resolution (by stating Before=pihole-FTL.service for its service), rather than vice versa?

Either way, rather than just defining a startup order (which applies only when both services are scheduled to start), I would also suggest making that dependency explicit by adding a Wants= option (either for pihole-FTL.service or wg-quick@wg0.service, whichever way you end up chosing your dependency to be).
This would not only affect system startup, but may help when manually starting and stopping services as well.

Hmm, based on our tests it seems not for every query, but the interface info needs to be somehow updated. E.g. if pihole-FTL starts before any network interface has been configured, no query is accepted (as local request) until dnsmasq update is triggered, which e.g. is done when opening the web UI or running pihole status, although not 100% sure why/what exactly is the trigger :sweat_smile:.

Good point. However a firewall seems to be not the reason since only switching to "all interfaces from all origins" solves the issue.

It is a VPN server (?), hence starting it does not require DNS but the other way round, dnsmasq requires all interfaces to be configured to detect correctly local IPs. I just found @DanSchaper verifying this which verifies as well that dnsmasq does not re-estimates available interfaces on every query: Have to "pihole restartdns" after reboot - #15 by DanSchaper

I am pretty sure that the 255.255.255.255 WireGuard netmask is the issue here which marks the server IP itself as only member of the subnet, hence the only valid "local origin". But let's wait for OP reply.

wg0.conf btw would need to contain Address = 10.0.40.6/24 for this while it currently seems to contain Address = 10.0.40.6/32.

None of these two trigger any sort of update. Both are really just passive activities.

Opening the web UI might involve resolving pi.hole, but that's it. pihole status involves sending an empty package (not even a valid DNS packet!) to port 53. That shouldn't do anything either:

Re-reading your statement,

you might have meant that FTL does not magically acquire new interfaces but only after a query has been made (from an already accepted interface!) before any queries from a new interface are accepted. This might be the case and may be a bug in dnsmasq. Users participating in the beta testing could check of FTL's branch update/dnsmasq to get the bleeding-edge dnsmasq code. This may or not resolve issues.

1 Like

Thank you all for your support.

@MichaIng was right. PICNIC :see_no_evil:

I was just copy-pasting and not thinking - ended up with Address = 10.0.40.6/32 changed it to 24 and it worked immediately.

Was looking for that but didn't occur (probably because the network existed - but was rather small).

Wouldn't that break both if one failed to start? I would prefer at least one service to run because this should be 'road warrior setup'. If pihole fails I can use wireguard to remotely log in and fix it. If wireguard fails it's not a big deal because my parents still get their ads filtered :slight_smile:

1 Like

Ah yeah that might be it: permitted/accepted queries only. I also saw the empty package sent, and to me it seems to be the only possibility that might trigger dnsmasq to update its interfaces info. I rechecked the code commit state of when we did those tests and there it was the same, echo > /dev/tcp/127.0.0.1/53 as only step that fits. This is not a DNS request, but since it comes from localhost it is at least from a permitted/accepted origin. Probably this is enough then for dnsmasq to update interface info. However deep insights missing from my side :wink:.

@yubiuser
Great it worked, so we now know for sure how dnsmasq checks for local origins :+1:. Strange that there was no related output, did you check /var/log/pihole.log for it?

I thought the same. Adding a strict dependency makes things inflexible. pihole-FTL does not require WireGuard vice versa, only when both are enabled and should work nicely together WireGuard needs to start before pihole-FTL. After=/Before= ordering is hence everything that one should add. I am just not 100% sure now if that systemd drop-in config works as long as pihole-FTL is a sysvinit service. systemd generates a systemd service from it, but not sure if it includes drop-ins then as well. Didn't think about this in the other topic. However easy to test:

root@dietpi:/etc/init.d# mv /lib/systemd/system/rsync.service /lib/systemd/system/rsync.service.bak
root@dietpi:/etc/init.d# systemctl daemon-reload
root@dietpi:/etc/init.d# systemctl cat rsync
# /run/systemd/generator.late/rsync.service
# Automatically generated by systemd-sysv-generator

[Unit]
Documentation=man:systemd-sysv-generator(8)
SourcePath=/etc/init.d/rsync
Description=LSB: fast remote file copy program daemon
After=remote-fs.target
After=nss-lookup.target
After=autofs.service

[Service]
Type=forking
Restart=no
TimeoutSec=5min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
SuccessExitStatus=5 6
ExecStart=/etc/init.d/rsync start
ExecStop=/etc/init.d/rsync stop
ExecReload=/etc/init.d/rsync reload
root@dietpi:/etc/init.d# mkdir /etc/systemd/system/rsync.service.d
root@dietpi:/etc/init.d# echo -e '[Service]\nExecStartPre=/bin/echo "It does!"' > /etc/systemd/system/rsync.service.d/test.conf
root@dietpi:/etc/init.d# systemctl daemon-reload
root@dietpi:/etc/init.d# systemctl cat rsync
# /run/systemd/generator.late/rsync.service
# Automatically generated by systemd-sysv-generator

[Unit]
Documentation=man:systemd-sysv-generator(8)
SourcePath=/etc/init.d/rsync
Description=LSB: fast remote file copy program daemon
After=remote-fs.target
After=nss-lookup.target
After=autofs.service

[Service]
Type=forking
Restart=no
TimeoutSec=5min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
SuccessExitStatus=5 6
ExecStart=/etc/init.d/rsync start
ExecStop=/etc/init.d/rsync stop
ExecReload=/etc/init.d/rsync reload

# /etc/systemd/system/rsync.service.d/test.conf
[Service]
ExecStartPre=/bin/echo "It does!"

It does :+1:.

Yes I did. Used the command from @Bucking_Horn, got no output.

edit

Switching DEBUG_ALL=true in pihole-FTL.conf and restoring /32 address also doesn't show any "ignoring" but

21886:[2020-04-06 20:36:37.309 5789] Interface lo is IPv4
21887:[2020-04-06 20:36:37.309 5789] Interface eth0 is IPv4
21888:[2020-04-06 20:36:37.309 5789] Interface wlan0 is IPv4
21889:[2020-04-06 20:36:37.309 5789] Interface lo is IPv4
21890:[2020-04-06 20:36:37.309 5789] Interface eth0 is IPv4
21891:[2020-04-06 20:36:37.309 5789] Interface wg0 is IPv4
21892:[2020-04-06 20:36:37.309 5789] Interface lo is IPv6
21893:[2020-04-06 20:36:37.309 5789] Interface eth0 is IPv6
21894:[2020-04-06 20:36:37.309 5789] Found 6 IPv4 and 2 IPv6 capable interfaces

Interesting, however based on "all origins" workaround and "/24" netmask solution there is doubt now that it was dnsmasq denying the query as "no local origin".

This confirms that pihole-FTL / dnsmasq wasn't ignoring some of your local subnets, suggesting another cause instead.
And indeed, as you've found out, you were applying an oversized routing prefix caused by an incorrect /32 netmask, leaving no room for any host identifier at all, not even a broadcast address (and in effect, no subnet).

No, Wants= is weaker than Requires=.

It will try to start the dependent unit, but will not fail itself if the dependent fails to start. (click for more)

From Raspbian's man systemd.unit:

Wants=
(..)
Units listed in this option will be started if the configuring unit is. However, if the listed units fail to start or cannot be added to the transaction, this has no impact on the validity of the transaction as a whole, and this unit will still be started. This is the recommended way to hook the start-up of one unit to the start-up of another unit.
(..)

Yes, must have been something else - I have no idea what and not the time to figure it out. I'm happy that it is working now.

That's true but here we really want before/after (depending on the use case *want* may be an option to

From freedesktop.org

Wants=
(...)
Note that requirement dependencies do not influence the order in which services are started or stopped. This has to be configured independently with the After= or Before= options. If unit foo.service pulls in unit bar.service as configured with Wants= and no ordering is configured with After= or Before= , then both units will be started simultaneously and without any delay between them if foo.service is activated.

Before= , After=
(...)
Note that those settings are independent of and orthogonal to the requirement dependencies as configured by Requires= , Wants= , Requisite= , or BindsTo= . It is a common pattern to include a unit name in both the After= and Wants= options, in which case the unit listed will be started before the unit that is configured with these options.

The key point is that wants units are started simultaneously by default, something we definitely not want here.

Agree, Wants is weaker than Requires but After is weaker then Wants and Wants is too strong and in fact simply not true for the dedicated service. All we want is: "If WireGuard is enabled/started anyway, wait for it to handle its interface correctly, else start up normally as WireGuard is disabled and Pi-hole does not need it and must not override the decision of the admin who disabled WireGard" :wink:.

Strange that "no subnet" is handled differently compared to "remote origin" then. I mean since both requests reach the process, and in fact both are handled with "all origins" allowed, remote origins must be dropped at the same point where "no subnet" requests are. Looks like a bug in dnsmasq to me then, if the one throws a meaningful log entry while the other is ignored silently. It simply make sense to handle both the same way: If a remote/www requests reaches dnsmasq, there is as well no matching local interface/subnet but an "isolated" public IP only which must then be (and is) dropped with "no local origin" log entry. Not urgent but probably worth to have dnsmasq upstream devs a look at to enhance debugging?

Yes, feel free to post to dnsmasq-discuss@lists.thekelleys.org.uk and ask for assistance. It might simply be a left-over nobody thought of trying when implementing this feature.

You seem to take my question(!) and its associated suggestion as a general recommendation preferring some options over others when it is not.

I was specifically referring to the "configurations used by you" (i.e, yubiuser) and used the exact words "suggest" as well as "whichever way you end up chosing" and left it for you to decide on that dependency question.

Likewise, my second reply to you (clarifying Wants=) was just that: A direct response to a question you asked.
I am not arguing either way, that's for you to decide - I am happy if you're happy. :wink:

Sorry if that caused some confusion.

I was just motivated by your answers to have a look what want` compared to after does. Because by intuition they seem equal to me: if one unit wants another it has to establish an order in which units are started, with the wanted to start first. Turns out my intuition was wrong and wants starts units simultaneously.

I just wanted to share this piece of information with fellow readers. My answer was also not intended for general recommendation but specific for my case (or others that might have trouble with interfaces not ready when pihole starts up).

I'm happy with my my decision - it works (at least for now) that's all I wanted. Hope your happy too :slight_smile:

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.