[bit.ly should open]
[My bit.ly redirect to 127.0.0.1 even though I have whitelisted both "bit.ly" in exact and regex " ^https?:\/\/bit\.ly\/(.+)$" . After whitelisting, I restarted DNS resolver. I am using Pihole with unbound.]
You have an exact whitelist entry for bit.ly and a regex whitelist entry for the string you list above. The string is not a valid domain regex, so the only rule in operation is the exact whitelist.
Try removing both rules and then adding a wildcard entry via Domains > Domain > [x] Add domain as wildcard and add bit.ly bitly.com. This will add regex entries for both, suitably configured for the domains plus any optional subdomains.
Does it work now? Also test from any computer which uses Pi-hole with the commands
nslookup bit.ly
nslookup bitly.com
These should return the IP addresses. If they were blocked they would return 0.0.0.0.
Thank you, it still doesn't seem to work unfortunately. Is there anything I can do to manually add a route?
Have added as you've explained:
(\.|^)bit\.ly$
(.|^)bitly.com$
pihole -t
20:18:55: query[A] bit.ly from 192.168.0.16
20:18:55: forwarded bit.ly to 127.0.0.1#5335
20:18:56: query[A] bit.ly from 192.168.0.16
20:18:56: forwarded bit.ly to 127.0.0.1#5335
20:18:58: query[A] bit.ly from 192.168.0.16
20:18:58: forwarded bit.ly to 127.0.0.1#5335
20:19:02: query[A] bit.ly from 192.168.0.16
20:19:02: forwarded bit.ly to 127.0.0.1#5335
NSLOOKUP:
C:\Users\XXXX>nslookup bit.ly
DNS request timed out.
timeout was 2 seconds.
Server: UnKnown
Address: 192.168.0.15
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
*** Request to UnKnown timed-out
C:\Users\XXXX>nslookup bitly.com
DNS request timed out.
timeout was 2 seconds.
Server: UnKnown
Address: 192.168.0.15
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
*** Request to UnKnown timed-out
This regex is not for a domain. This regex is trying to whitelist an URL, but Pi-hole works with domains.
The regex should never include the protocol (https://) part.
This is what query log is displaying;
|2023-11-16 20:24:50|HTTPS|bit.ly|XXXX.lan|OK (answered by localhost#5335)|SERVFAIL (7.5ms)
|2023-11-16 20:24:50|A|bit.ly|XXXX.lan|OK (answered by localhost#5335)|SERVFAIL (6.3ms)
|2023-11-16 20:24:50|HTTPS|bit.ly|XXXX.lan|OK (answered by localhost#5335)|SERVFAIL (3.7ms)||
========
XXXX.lan is my computer where I am trying to reach it from
Tried from a mobile phone also, same result.
Thank you, I fixed the bad regex following Chris advice, it still doesn't work.
In Pi-hole enable Settings > DNS > Use DNSSEC > Save. This will make Pi-hole display the DNSSEC results in the Query Log, eg INSECURE, SECURE, BOGUS, etc.
Check if the config file is happy
unbound-checkconf /etc/unbound/unbound.conf.d/pi-hole.conf
Flush previous Unbound results
sudo unbound-control flush_zone .
Run the nslookup commands again. Do you get the same SERVFAIL errors? Refresh Pi--hole's Query Log and the entries should be there. Are they reporting BOGUS?
nslookup shows the Server it used for the DNS request as unknown, when that would be expected to be pi.hole (or a simple repetition of the IP address).
This usually indicates something interfering with DNS requests in your network, potentially intercepting them.
Those time-outs are not correlating with Pi-hole's logs showing the reply to be SERVFAIL.
Not disregarding that SERVFAIL itself is unexpected here, but if that reply would have made it to the device issuing that nslookup, it would have displayed that reply instead of reporting a timeout.
Together, that is a strong indication that DNS requests are interfered with.
Likely candidates would be your router forcefully redirecting DNS requests, or some antivirus tools (e.g. AVG Secure DNS or AVAST Real-Site) on the device that issued the nslookup.
For the latter, those features have to be disabled when you make use of Pi-hole.
Would it always be resolved as pi.hole in the absence of anything overriding it? He mentioned it also failing from a mobile phone, but they can also run these AV clients, especially Android. Or else the router or ISP are involved.
@Raspbian can you try this command from your Windows terminal please (same place you ran the previous nslookup's)
nslookup -class=chaos -type=txt version.bind
I have disabled DHCP on ISP router, and set pi to DHCP so it can override ISP dns.
I can ping pi.hole from my windows client but can't nslookup it.
nslookup -class=chaos -type=txt version.bind
DNS request timed out.
timeout was 2 seconds.
Server: UnKnown
Address: 192.168.0.15
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
*** Request to UnKnown timed-out
=========
nslookup pi.hole
DNS request timed out.
timeout was 2 seconds.
Server: UnKnown
Address: 192.168.0.15
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
*** Request to UnKnown timed-out
ping pi.hole
Pinging pi.hole [192.168.0.15] with 32 bytes of data:
Reply from 192.168.0.15: bytes=32 time=10ms TTL=64
Reply from 192.168.0.15: bytes=32 time=10ms TTL=64
Reply from 192.168.0.15: bytes=32 time=13ms TTL=64
Reply from 192.168.0.15: bytes=32 time=12ms TTL=64
I tried again with av and firewall disabled, also from a mobile, i can't figure out. I've tried to find the issue in forums im not finding an answer unfortunately.
dig pi-hole.net @127.0.0.1 -p 5335
; <<>> DiG 9.16.44-Raspbian <<>> pi-hole.net @127.0.0.1 -p 5335
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18440
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;pi-hole.net. IN A
;; ANSWER SECTION:
pi-hole.net. 300 IN A 3.18.136.52
;; Query time: 19 msec
;; SERVER: 127.0.0.1#5335(127.0.0.1)
;; WHEN: Thu Nov 16 21:57:35 GMT 2023
;; MSG SIZE rcvd: 56
I think this can resolve now. I used a random bit.ly url that contained a forwarding "https://bitly.com/bbZkLH" and it works. Getting to just bit.ly doesn't work which is what i proceeded to test with after i realised bit.ly/XXXXXX wasn't working initially
The whitelisting guide provided by @chrislph must have been the thing that resolved the issue.
https://but.ly still re-directs to 127.0.0.1#5335 and page doesn't load, but I don't really care about reaching that domain without a shortened URL.... At least now any URL https://bit.ly/*xxxxx* works. Thank you for your help folks 
Also I think this might have helped from an earlier suggestion in addition with stopping and starting unbound and I also flush 24 hour query logs (that is all of them because it hasn't ran for more than 24 hours.)
sudo unbound-control flush_zone .
sudo service unbound stop
sudo service unbound start
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.