Pihole is installed on a VPS (Ubuntu 24.04). Access to port 53 is controlled via iptables.
Ads blocking works fine.
I can also access the web interface without issue. Yesterday I put it behind a nginx proxy with site SSL. Still no issue.
Actual Behaviour:
The problem (from before I installed the proxy server) is that queries don't seem to be recorded in the DB. The reason I say so is that when I click the query log from the web interface I just see hourly '1.0.0.127.in-addr.arpa' requests.
Also the dashboard doesn't show any clients query (although some clients are accessing it).
The debug log says the default gateway can't be ping. The VPS has a 10.x.x.x address and gateway and a fix public ip address (that wasn't detected by pihole).
Following up on the above I found that, while the gateway identified by pihole (10.128.0.2) is not pingable directly, the link-local address 169.254.169.254 is and I'm thinking that if I could tell pihole to use this link-local address rather than the current one this could resolve my issue.
You are right. Google Cloud (as AWS and other cloud services) uses this address to emulate a DHCP service.
I made a mistake above identifying 10.128.0.2 as the gateway. It rather is a static private address, and the gateway is 10.128.0.1. But this gateway isn't reachable while 169.254.169.254 is.
Not sure if I can change the network configuration since this is a cloud VPS. This is why I was looking for a way to "push" a gateway address to pihole.
More on the VPS networking (from Pihole debug):
*** [ DIAGNOSING ]: Network interfaces and addresses
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc mq state UP group default qlen 1000
link/ether 42:01:0a:80:00:02 brd ff:ff:ff:ff:ff:ff
inet 10.128.0.2/32 metric 100 scope global dynamic ens4
valid_lft 3211sec preferred_lft 3211sec
inet6 fe80::4001:aff:fe80:2/64 scope link
valid_lft forever preferred_lft forever
*** [ DIAGNOSING ]: Network routing table
default via 10.128.0.1 dev ens4 proto dhcp src 10.128.0.2 metric 100
10.128.0.1 dev ens4 proto dhcp scope link src 10.128.0.2 metric 100
169.254.169.254 via 10.128.0.1 dev ens4 proto dhcp src 10.128.0.2 metric 100
*** [ DIAGNOSING ]: Networking
[i] Default IPv4 gateway(s):
10.128.0.1%ens4
* Pinging first gateway 10.128.0.1...
[✗] Gateway did not respond. (https://discourse.pi-hole.net/t/why-is-a-default-gateway-important-for-pi-hole/3546)
Not being able to set a static IP could cause you problems in the future, I'd see if there's any way to set that. It may be in their website.
It looks like the pi-hole is set to DHCP still, I'd be surprised if they give you a wrong DHCP lease. It's possible they drop pings to the default gateway. Can you try ping 1.1.1.1 on the VPS and see if you have connectivity?
My understanding (and I could very well be wrong) is that my local ip 10.128.0.2 is routed via the 169.254.169.254 link-local address to a static public IP (34.n.n.n).
I can ping 1.1.1.1 or any other ip address from the VPS and I can reach the VPS using this public ip, with respect to the firewall rules of course.
default route is via 10.128.0.1. Even pinging 169.254.169.254 will go through 10.128.0.1. 169.254.169.254 via 10.128.0.1 dev ens4 proto dhcp src 10.128.0.2 metric 100
If your VPS has internet connectivity, I'd be pretty sure that pings are just disabled on the gateway.
Edit: I've not seen any issues with pi-hole not recording DNS queries. I'm willing to bet this is a network issue. How are you connecting to your pi-hole? Through the public IP?
I agree, pings to the gateway (10.128.0.1) are probably just disabled.
I am also with you thinking that it is not that pihole doesn't log to the db, it's rather that it doesn't see these requests, which points to a network issue. Clients access pihole via the VPS' public ip on port 53 and this part works well with good responses and blocking ads.
A sql query to the database shows almost only PTR requests, see below. There are very few exceptions. all IPv6 like the one from px.za.zaloapp.com below. Even when I query the db, just after accessing a new site (to exclude caches) from my browser, nothing appears in the db:
sudo sqlite3 /etc/pihole/pihole-FTL.db
SQLite version 3.45.1 2024-01-30 16:01:20
Enter ".help" for usage hints.
sqlite> select * from queries;
1|1744242006.32129|6|3|1.0.0.127.in-addr.arpa|127.0.0.1|||5|0.000224828720092773|0||-1
2|1744242008.26436|2|1|px.za.zaloapp.com|fe80::4001:aff:fe80:2||-3|4|0.00367903709411621|0|-3|-1
3|1744242008.21428|2|1|px.za.zaloapp.com|::1||-3|4|0.00693655014038086|0|-3|-1
4|1744243200.63309|6|3|2.0.0.0.0.8.e.f.f.f.a.0.1.0.0.4.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa|127.0.0.1|||6|0.000158309936523438|0||-1
5|1744243200.62528|6|3|1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa|127.0.0.1|||5|0.00750041007995605|0||-1
6|1744243200.62489|6|3|1.0.0.127.in-addr.arpa|127.0.0.1|||5|3.40938568115234e-05|0||-1
7|1744246800.58551|6|3|1.0.0.127.in-addr.arpa|127.0.0.1|||5|0.0001373291015625|0||-1
8|1744250400.52722|6|3|1.0.0.127.in-addr.arpa|127.0.0.1|||5|0.000100851058959961|0||-1
9|1744254000.49371|6|3|1.0.0.127.in-addr.arpa|127.0.0.1|||5|0.000196456909179688|0||-1
10|1744257600.4639|6|3|1.0.0.127.in-addr.arpa|127.0.0.1|||5|0.000199079513549805|0||-1
11|1744261200.40415|6|3|1.0.0.127.in-addr.arpa|127.0.0.1|||5|0.000397443771362305|0||-1
12|1744264800.36128|6|3|1.0.0.127.in-addr.arpa|127.0.0.1|||5|0.000171899795532227|0||-1
13|1744268400.30798|6|3|1.0.0.127.in-addr.arpa|127.0.0.1|||5|4.38690185546875e-05|0||-1
14|1744272000.2616|6|3|1.0.0.127.in-addr.arpa|127.0.0.1|||5|0.000169992446899414|0||-1
15|1744275600.20579|6|3|1.0.0.127.in-addr.arpa|127.0.0.1|||5|6.93798065185547e-05|0||-1
...
I would strongly recommend not using pi-hole as an open resolver to the internet. Consider using something like Wireguard to connect your VPS to your home LAN and serve pi-hole through there. This is infinitely more secure.
With that said, let's confirm you're even using pi-hole for your DNS resolution. Please run nslookup pi.hole <VPS public IP> from a client device.
I do understand your recommendation relative to an open resolver on the internet and this server is not, since its access is tightly controlled via the firewall.
Here is the output of the nslookup (the public ip address has been changed below for security reasons):
Querying pi.hole should yield a response with it's own private IP address. Since you received an nxdomain I'm willing to bet that you are not actually querying your pi-hole. using the command nslookup pi.hole 127.0.0.1 on the VPS should yield a response.
Sometimes VPS providers will use unconventional network design to meet their needs. Are they using a CG-NAT? My best guess here is that you are using someone else's open resolver on that VPS hosting platform and they so happen to block similar domains.
From the Network routing table (see above) I knew that 169.254.169.254 was resolving names but I thought that this was for the ones that pihole did not block and sent up to my upstream resolvers (1.1.1.1).
However the tests we just did seem to tell a different story. Just to confirm I added a custom domain I own, with the exact deny action. Querying this domain from 127.0.0.1 returned me an address of 0.0.0.0 (which is right) but querying this same custom domain from an allowed client using the VPS public IP returned the actual address of this custom domain (so pihole wasn't involved).
All of that to say that you are right that my VPS ignores pihole and uses a Google open resolver that block similar domains. Traffic to port #53 is probably intercepted even before reaching my VPS. I'll have to get to Google admins to see if they can let me manage this port.
