Adlist connections refused, no cached list downloaded

dgani · November 26, 2023, 4:09am

Expected Behaviour:

When Update Gravity is run, the pi should be able to connect to the website so that it can create a cached list.

Actual Behaviour:

All tested lists have had their connections refused.

Debug Token:

~~https://tricorder.pi-hole.net/TNL0r8IG~~
https://tricorder.pi-hole.net/TNLOr8IG/

Additional Info:

The three lists I have tested with are
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://adaway.org/hosts.txt
https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts.txt

The output of pihole -g -f is

Status: Connection Refused
List download failed: no cached list available

For all the above targets.

chrislph · November 26, 2023, 7:35am

Can you try the command again please, as it may have been a short outage on your connection?

This looks like a new install as of yesterday, is that right? What is the exact OS and version it was installed on?

dgani · November 26, 2023, 5:25pm

Yes, this is a new install. My pi is a bit old, so maybe that's the issue!

Here is the /etc/os-release:

NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

Trying gravity again...
pi hole diagnosis lists them all as inaccessable again.|

Output of pihole -g -f:

  [✓] Deleting existing list cache
  [i] Neutrino emissions detected...
  [✓] Pulling blocklist source list into range

  [✓] Preparing new gravity database
  [✓] Creating new gravity databases
  [i] Using libz compression

  [i] Target: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available

  [i] Target: https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts.txt
  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available

  [i] Target: https://adaway.org/hosts.txt
  [✗] Status: Connection Refused
  [✗] List download failed: no cached list available

  [✓] Building tree
  [✓] Swapping databases
  [✓] The old database remains available
  [i] Number of gravity domains: 0 (0 unique domains)
  [i] Number of exact blacklisted domains: 0
  [i] Number of regex blacklist filters: 0
  [i] Number of exact whitelisted domains: 0
  [i] Number of regex whitelist filters: 0
  [✓] Flushing DNS cache
  [✓] Cleaning up stray matter

  [✓] FTL is listening on port 53
     [✓] UDP (IPv4)
     [✓] TCP (IPv4)
     [✓] UDP (IPv6)
     [✓] TCP (IPv6)

  [✓] Pi-hole blocking is enabled

chrislph · November 26, 2023, 5:38pm

What are the permissions on /etc/pihole?

ls -ld /etc/pihole

I think it will need a dev to advise on this debug log.

dgani · November 26, 2023, 5:48pm

drwxrwxr-x 3 pihole pihole 4096 Nov 26 09:48 /etc/pihole

DanSchaper · November 26, 2023, 6:02pm

Can you run a few commands for us?

curl -Li https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts | head -n 20

And

curl -L https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
echo $?

The first will show the headers (if any) that are being passed and the second will show the exit code for the command, if it fails again then we can see if the reason is a common one.

dgani · November 26, 2023, 7:03pm

curl -Li https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts | head -n 20
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:15 --:--:--     0HTTP/2 200 
cache-control: max-age=300
content-security-policy: default-src 'none'; style-src 'unsafe-inline'; sandbox
content-type: text/plain; charset=utf-8
etag: "74cf6c078867e75d311e11633db2e058b4ac4c495904955d27d1890880ad598d"
strict-transport-security: max-age=31536000
x-content-type-options: nosniff
x-frame-options: deny
x-xss-protection: 1; mode=block
x-github-request-id: B338:1DCE:962202:C1BE3C:65637468
accept-ranges: bytes
date: Sun, 26 Nov 2023 18:58:15 GMT
via: 1.1 varnish
x-served-by: cache-sjc10065-SJC
x-cache: HIT
x-cache-hits: 1
x-timer: S1701025096.566919,VS0,VE4
vary: Authorization,Accept-Encoding,Origin
access-control-allow-origin: *
cross-origin-resource-policy: cross-origin
  0 4799k    0 48211    0     0   3183      0  0:25:43  0:00:15  0:25:28 11880
curl: (23) Failed writing body (61 != 16375)

and the second command outputs the website's list of domains (too long to post)

DanSchaper · November 26, 2023, 7:37pm

And pihole -g still fails?

dgani · November 26, 2023, 7:55pm

Yeah it still fails.

DanSchaper · November 26, 2023, 8:26pm

Hmm, okay, if you don't mind doing some investigating work:

Can you run

sudo bash -x /opt/pihole/gravity.sh

And watch the output, if it's still failing then the reasons should be in the output. I'm guessing it will be after the database routines, which look like CREATE TRIGGER. Might be in the httpCode.

This is what a successful run looks like:

++ curl --connect-timeout 10 -s -L --compressed -z /etc/pihole/list.1.raw.githubusercontent.com.domains -w '%{http_code}' https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts -o /tmp/tmp.dK9SvwkgKF
+ httpCode=200
+ case $url in
+ case "${httpCode}" in
+ echo -e '\r  [✓] Status: Retrieval successful'

dgani · November 27, 2023, 11:15pm

Not 100% sure what to look for - the command returned a lot of information. But I piped it into the attached file.

Command ran:

sudo bash -x /opt/pihole/gravity.sh &> /home/pi/Desktop/xlog.txt

xlog.txt (20.5 KB)

DanSchaper · November 28, 2023, 5:14pm

Thanks for the upload.

Can you run the following for me?

curl --connect-timeout 10 -s -L --compressed -w '%{http_code}\n' https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts -o /tmp/tmp.DZE2Jn7Rrg ; echo "Exited with $?"

Your results show a timeout httpCode=000 value, so there's something in the way between Pi-hole's calls and the external servers.

If you want to give some additional information then can you run

curl --connect-timeout 10 -s -L --compressed -w '%{http_code}\n' https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

And see if there is any response at all from github? And if you do get the proper list shown can you please show the last few lines of the output, like:

# 'hosts' file format) are appended to the unified hosts file during the
# update process. For example, uncomment the following line to block
# 'example.com':

# 0.0.0.0 example.com
200

That final line, 200 in a good run, is what I am looking for.

It may be possible that writing the file to the /tmp directory is the issue and the output from the commands I have shown above will help us narrow down the possibilities.

dgani · November 30, 2023, 3:04am

I have no idea what happened, but it's working now.

I think it may have been an issue with my ISP? Honestly my least favorite type of conclusion to a problem

Thank you for all your help!

DanSchaper · November 30, 2023, 5:27pm

Oh, I get you there.

Glad it's working. Please link to this thread if the issue comes up again and you need to open a new topic.

Thanks for the follow up!

sean311 · December 8, 2023, 8:04am

I have same problem

[Fri Dec 08] root@sandalpi: ~$ pihole -v
  Pi-hole version is v5.17.2 (Latest: v5.17.2)
  web version is v5.21 (Latest: v5.21)
  FTL version is v5.23 (Latest: v5.23)
[Fri Dec 08] root@sandalpi: ~$

Hope you can shed some light on it please ?
I have attached output of pihole -g -f

pihole-g-f-output.txt (10.2 KB)

DanSchaper · December 8, 2023, 5:38pm

Please open a new topic, filling out the template in full.

You have some extensive corruption in your configuration.

sean311 · December 9, 2023, 12:25am

Hi @DanSchaper thanks for reply. I have done that.

Please help

system · December 30, 2023, 12:26am

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.