Hello!
I have freshly installed Pi-hole and have tried to point it to several available upstream variants:
Dnscrypt running on same system
Google dns
Local router
Every time I try to resolve something using pi-hole, response timeouts. Pihole itself logs forwarding that request to upstream, but doesn't log getting any result, so it shows N/A in admin panel.
If I try to resolve blacklisted or local name, pi-hole returns 0.0.0.0 or local address respectively, so it does work that way at least.
I'm running latest xbian on Raspberry Pi 3B and did install with "ignore os check" flag, because installer reported rasbpian 1 as unsupported. Also After install pihole-FLT could not run due to library it is dependant on being in another location. After copying /lib/arm-linux-gnueabihf/ld-linux.so.3 to /lib/, FTL started successfully.
DNSCrypt is running on port 5353 and resolves any request I point to it correctly. Resolving to 8.8.8.8 or my router also works.
Pihole-FTL is listening on port 53 on all interfaces.
; <<>> DiG 9.11.5-P4-5.1+deb10u2-Debian <<>> example.com 127.0.0.1
;; global options: +cmd
;; connection timed out; no servers could be reached
I spent most of the day trying to ascertain reason for such behavior, but could not get it. Maybe I'm doing something wrong?
My debug token is 7ka6o7jmvq
Mar 7 10:21:44 dnsmasq[7843]: query[A] example.com from 127.0.0.1
Mar 7 10:21:44 dnsmasq[7843]: forwarded example.com to 127.0.0.1
Mar 7 10:21:49 dnsmasq[7843]: query[A] example.com from 127.0.0.1
Mar 7 10:21:49 dnsmasq[7843]: forwarded example.com to 127.0.0.1
Mar 7 10:21:54 dnsmasq[7843]: query[A] example.com from 127.0.0.1
Mar 7 10:21:54 dnsmasq[7843]: forwarded example.com to 127.0.0.1
so your Pi-hole is receiving and forwarding the queries to the configured upstream destination (ports aren't shown here) but it never receives a reply. Do you have any kind of firewall (ufw, iptables, etc.) in place that could limit any traffic?
Others than that, the statistics of your Pi-hole shows that it worked at some point:
[2021-03-07 03:06:08.029 7839M] Imported 2351 queries from the long-term database
[2021-03-07 03:06:08.030 7839M] -> Total DNS queries: 2351
[2021-03-07 03:06:08.030 7839M] -> Cached DNS queries: 181
[2021-03-07 03:06:08.030 7839M] -> Forwarded DNS queries: 2157
[2021-03-07 03:06:08.030 7839M] -> Blocked DNS queries: 13
[2021-03-07 03:06:08.030 7839M] -> Unknown DNS queries: 0
[2021-03-07 03:06:08.030 7839M] -> Unique domains: 35
[2021-03-07 03:06:08.030 7839M] -> Unique clients: 3
[2021-03-07 03:06:08.030 7839M] -> Known forward destinations: 6
Also, your DHCP server does not announce a DNS server at all, but this can be addressed later.
I do not have any firewall on Pi set up. The only firewall I have is on my router - and it only blocks DNS queries from internet to it's WAN. Also tcpdump shows that I do get responses (I point to google as upstream for this test), but for some reason it shows that udp port is unreachable on pihole side
tcpdump host 8.8.8.8
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wlan0, link-type EN10MB (Ethernet), capture size 262144 bytes
11:18:18.971119 IP 192.168.88.66.59133 > dns.google.domain: 58961+ A? example.com. (29)
11:18:19.023050 IP dns.google.domain > 192.168.88.66.59133: 58961 1/0/0 A 93.184.216.34 (45)
11:18:19.023212 IP 192.168.88.66 > dns.google: ICMP 192.168.88.66 udp port 59133 unreachable, length 81
11:18:20.977787 IP 192.168.88.66.55201 > dns.google.domain: 25761+ AAAA? example.com. (29)
11:18:21.036975 IP dns.google.domain > 192.168.88.66.55201: 25761 1/0/0 AAAA 2606:2800:220:1:248:1893:25c8:1946 (57)
11:18:21.037134 IP 192.168.88.66 > dns.google: ICMP 192.168.88.66 udp port 55201 unreachable, length 93
11:18:22.980371 IP 192.168.88.66.21552 > dns.google.domain: 31729+ A? example.com. (29)
11:18:23.028452 IP dns.google.domain > 192.168.88.66.21552: 31729 1/0/0 A 93.184.216.34 (45)
11:18:23.028646 IP 192.168.88.66 > dns.google: ICMP 192.168.88.66 udp port 21552 unreachable, length 81
11:18:24.984918 IP 192.168.88.66.39401 > dns.google.domain: 49647+ AAAA? example.com. (29)
11:18:25.041938 IP dns.google.domain > 192.168.88.66.39401: 49647 1/0/0 AAAA 2606:2800:220:1:248:1893:25c8:1946 (57)
11:18:25.042117 IP 192.168.88.66 > dns.google: ICMP 192.168.88.66 udp port 39401 unreachable, length 93
Pi-hole never worked for me - all the queries that were forwarded, have response N/A on them and blocked queries are the ones that I have tested with nslookup pointed to pihole.
DCHP server is giving out provider DNS and it is showing up in /etc/resolv.conf
Thanks for the recording, this is showing the issue: Your Pi-hole is asking Google's server from UDP port 59133 which is also where Google is responding to. However, there is something rejecting the packets with ICMP PORT UNREACHABLE :
What is the output of
sudo iptables -L -n
?
This isn't reflected in your debug log. Try
sudo pihole-FTL dhcp-discover
and check if there are dns-server lines in the DHCP options (there were none in your debug log).
As for DHCP (although I don't think it really matters, since pihole ignores system dns, right?):
pihole-FTL dhcp-discover
Scanning all your interfaces for DHCP servers
Timeout: 10 seconds
Received 300 bytes from wlan0:192.168.88.1
Offered IP address: 192.168.88.66
Server IP address: 192.168.88.1
Relay-agent IP address: N/A
BOOTP server: (empty)
BOOTP file: (empty)
DHCP options:
Message type: DHCPOFFER (2)
server-identifier: 192.168.88.1
lease-time: 86400 ( 1d )
--- end of options ---
DHCP packets received on interface eth0: 0
DHCP packets received on interface lo: 0
DHCP packets received on interface wlan0: 1
So I'm a bit running out of ideas right now. Is there anything you have installed on this system next to Pi-hole? I tried to reproduce your setup as closely as possible (I cannot use a WiFi right now, but you'll not be the first one with this setup). However, I don't see the ICMP error and everything is working correctly. I'll try to think about it a bit more. Maybe someone else will chime in and have some further idea what to look out for meanwhile.
Yeah, its separate, just wanted to point it out as I've seen it to ensure we won't forget it once we solved the other issue (as no DNS information supplied via DHCP could be causing other issues).
Yeah, I'm at a loss too, to be frank.
Pi-hole does not has much currently: openvpn client with static routes to some (not all) ip-addresses.
At least I can confirm that traffic to 8.8.8.8 is being routed via my router and does not go into tunnel.
It also has kodi and I mentioned dnscrypt, whic is working fine. I even tried to put dnscrypt on port 53 instead of pihole and everything is resolved as needed. I also tried disabling dnscrypt in case it's messing with something and leave pihole with google dns. Nothing changed.
It looks like socket is closed right after making request, hence port being unavailable to receive response. Or is it impossible? I don't know how to check this theory.
I tried right now ethernet interface instead of wifi (reconfigured pihole accordingly) and got same results.
Also same ICMP error is present on loopback interface (dnscrypt).
What you could try is stopping pihole-FTL and starting dnsmasq (you may need to install it for this purpose). It will use the same config files FTL uses for the DNS part.
If you can confirm the same behavior with dnsmasq then it's clearly a bug with dnsmasq and we should report it to the mailing list so it gets fixed (we will inherit the fix into Pi-hole).
There has been a change with regards to the ports used for upstream communication rather recently (1-2 months ago) and since FTL always incorporates the latest version, this may be the next thing we should look at (in case the system-provided dnsmasq operates as expected).
To rule this out, we may need to compile the latest dnsmasq from source for comparison. We can do this together, if you like when it looks like we have to.
Tried dnsmasq and it works.
Tcpdump still shows port unreachable, but dns queries are being resolved just fine.
Pihole shows dns server and FTL being off and doesn't block anything though, I guess it's normal, since it needs pihole-FTL?
So it'd be very helpful if you could install dnsmasq from source so we can see if the issue appeared between v2.80 and v2.84 or if there is really a difference caused by the Pi-hole modifications of dnsmasq.
Try
git clone http://thekelleys.org.uk/git/dnsmasq.git
cd dnsmasq
git checkout v2.84
make COPTS="-DHAVE_DNSSEC -DHAVE_DNSSEC_STATIC -DHAVE_IDN"
sudo make install
sudo service dnsmasq restart
If you need to install dependencies because the build fails, try installing them as suggested here.
After the build succeeded,
./src/dnsmasq -v
should result in
Dnsmasq version 2.84 Copyright (c) 2000-2021 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth cryptohash DNSSEC loop-detect inotify dumpfile
You can always query the currently running version using
Interesting. I tried to do the same but also don't see the issue. I wonder if it has to do with dnsmasq running as root while FTL is running as unprivileged user pihole.
@Demoneeho Could you also try building FTL from source and check if this misbehaves the same way as the provided FTL binary? The page linked above by @DL6ER (with the dependencies) contains how to do it.
So it works with pihole-FTL compiled from source but not with pre-compiled pihole-FTL downloaded from Github? This is weird but okay. @DL6ER check this out!
As I said earlier, it's latest xbian (which should be based on Debian Buster) installed on RPi 3B.
I didn't notice binary being wrong version and that's probably what caused problems, since after I compiled FTL from source everything works as intended.
Hmm, interesting. Even when the armv5te binary should work on an armv7l chip without any issues, this may not be (always) true in practice. Good catch!
The issue likely comes from this line:
From the given ldd output, this will result in
/lib/libarmmem.so
/lib/ld-linux-armhf.so.3
which is not recognized by the installer so, instead of continuing the chain detecting armhf and then v7, it just picks ARMv5TE which is the fallback when the CPU is not explicitly ARMv4.
ARMv5TE is the fallback because (a) ARMv5TE allows for a lot more optimization than v4 does and (b) v5 (1997) is found a lot more often than v4 (standardized 1993 !).
Without any further testing right now, I'd say the fix will be to change
like
l_binary="pihole-FTL-aarch64-linux-gnu"
#
-elif [[ "${lib}" == "/lib/ld-linux-armhf.so.3" ]]; then
+elif [[ "${lib}" == *"/lib/ld-linux-armhf.so.3"* ]]; then
# Hard-float available: Use gnueabihf binaries
# If ARMv8 or higher is found (e.g., BCM2837 as found in Raspberry Pi Model 3B)
if [[ "${rev}" -gt 7 ]]; then
