Please follow the below template, it will help us to help you!
Expected Behaviour:
Successfully download a list from a blocked domain.
Actual Behaviour:
It fails all the time. I need to whitelist the domain before, then it works.
Here an example output from gravity update:
[i] Target: s3.amazonaws.com (simple_malvertising.txt)
[✗] Status: s3.amazonaws.com is blocked by . Using DNS on 8.8.8.8 to download https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt
[✗] Status: https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt (304000)
[✗] List download failed: using previously cached list
Did i miss something or is this maybe broken?
Pi-hole Version: v4.3.1
I'm not a specialist, but I would say that Pi-hole does exactly what it was taught. Namely, the domains contained in block lists to block the DNS queries.
Search via "Query Lists" in the web interface in which blocklist the domain s3.amazonaws.com is contained or if a regex is responsible for the blocking. That you can - I think - with "Tail Pi-hole.log" find out in the admin interface of your Pi-hole.
Then you can decide whether you remove the domain Whitlistest or the responsible blocklist.
Pi-Hole is working as intended. Line 2 of your output - Pi-Hole recognizes that the domain is blocked, so it bypasses itself and tries to load the list using your specified upstream DNS server directly.
The list would not load from the requested server.
Just curious - why do you have s3.amazonaws.com blocked?
I'm just using several lists i found here in this forum. Are you sure, that this is intended? Pi-hole detect the dns block, which is correct and it's using 8.8.8.8 to resolve s3.amazonaws.com. So to get the ip and then download the file shouldn't be the problem, right? Or maybe i just misinterpret this message
Pi-hole is seeing that the list source is blocked and then uses 8.8.8.8 to resolve the IP of the target list. That all appears to be functioning as it should. What looks like is happening is that either the target URL is gone or there is something else preventing you from downloading that URL.
A quick check would be to set the /etc/resolv.conf to 8.8.8.8 on the Pi-hole and then trying to curl that file. If that fails then you should get a reason for the failure to download. Just set the /etc/resolv.conf back to what it was previously when you are done.
pi@raspberrypi:~ $ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
pi@raspberrypi:~ $ pihole -g
[i] Pi-hole blocking is enabled
[i] Neutrino emissions detected...
[✓] Pulling blocklist source list into range
[i] Target: raw.githubusercontent.com (hosts)
[✓] Status: Retrieval successful
[i] Target: mirror1.malwaredomains.com (justdomains)
[✓] Status: No changes detected
[i] Target: sysctl.org (hosts)
[✓] Status: No changes detected
[i] Target: zeustracker.abuse.ch (blocklist.php?download=domainblocklist)
[✓] Status: No changes detected
[i] Target: s3.amazonaws.com (simple_tracking.txt)
[✗] Status: s3.amazonaws.com is blocked by . Using DNS on 8.8.8.8 to download https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
[✗] Status: https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt (200000)
[✗] List download failed: using previously cached list
Do you have any regex blocks added? You can run sudo bash /opt/pihole/gravity.sh to see the output, looking for a line that has httpCode as something not 200 or 304.
What was the result/response from the curl? If you run sudo bash /opt/pihole/gravity.sh you'll get the full display with all of the result variables and string checks to see why the script is showing a failure.
Blocking the entire s3 infrastructure is overkill but you can find the exact name of the list blocking via:
pi@raspberrypi:~ $ sudo bash /opt/pihole/gravity.sh
[i] Pi-hole blocking is enabled
[i] Neutrino emissions detected...
[✓] Pulling blocklist source list into range
[i] Target: raw.githubusercontent.com (hosts)
[✓] Status: Retrieval successful
[i] Target: mirror1.malwaredomains.com (justdomains)
[✓] Status: No changes detected
[i] Target: sysctl.org (hosts)
[✓] Status: No changes detected
[i] Target: zeustracker.abuse.ch (blocklist.php?download=domainblocklist)
[✓] Status: No changes detected
[i] Target: s3.amazonaws.com (simple_tracking.txt)
[✗] Status: s3.amazonaws.com is blocked by . Using DNS on 8.8.8.8 to download https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
[✗] Status: https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt (200000)
[✗] List download failed: using previously cached list
[i] Target: s3.amazonaws.com (simple_ad.txt)
[i] Status: Pending...^C
[i] User-abort detected
[✓] Cleaning up stray matter
[✓] DNS service is running
[✓] Pi-hole blocking is Enabled
pi@raspberrypi:~ $ pihole -g
[i] Pi-hole blocking is enabled
[i] Neutrino emissions detected...
[✓] Pulling blocklist source list into range
[i] Target: raw.githubusercontent.com (hosts)
[✓] Status: Retrieval successful
[i] Target: mirror1.malwaredomains.com (justdomains)
[✓] Status: No changes detected
[i] Target: sysctl.org (hosts)
[✓] Status: No changes detected
[i] Target: zeustracker.abuse.ch (blocklist.php?download=domainblocklist)
[✓] Status: No changes detected
[i] Target: s3.amazonaws.com (simple_tracking.txt)
[✗] Status: s3.amazonaws.com is blocked by . Using DNS on 8.8.8.8 to download https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
[✗] Status: https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt (200000)
[✗] List download failed: using previously cached list
[i] Target: s3.amazonaws.com (simple_ad.txt)
[✗] Status: s3.amazonaws.com is blocked by . Using DNS on 8.8.8.8 to download https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
[i] Status: Pending...^C
[i] User-abort detected
[✓] Cleaning up stray matter
[✓] DNS service is running
[✓] Pi-hole blocking is Enabled
Result of the curl. I've added single quotes for the useragent string, which was missing from my echo output. It's strange that there is HTML in the tmp file.
