Support local files in adlist.list


#1

Recently I stumbled over a hosts file I wanted to include in my pi-hole. The problem is: The hosts file is not provided in text form, only as a zipped file. So I couldn’t just add the URL to my adlist.list, but had to download and extract the file manually.

Now, I could just add this hosts file to dnsmasq.conf, but then it’s not ‘integrated’ with pi-hole. That’s why I tried to add it throuh the adlist.list file with file:///path/to/file.

The problems with this:

  • The curl works, but because the http response code is 000 instead of 200, the gravity script considers it as error (even though the curl works just fine)

  • You can get it work by renaming the file to the “pihole lists format”, but the name has to be “” [empty], probably because the name is read between the second / and the third / (https://domainname.com/), but the local address in adlists.list is file:/// (3 slashes).

Could you implement the recognition of a local file (with file://) in the gravity script so that the name is not empty and the http resonse code 000 is a success for the local file?
I think this is probably a few minutes thing if you’re familiar with the gravity script code.

Or could there be an easier solution for importing zipped hosts file into pi-hole?

Thank you!


#2

I have seen a few people asking for this, too. For now, we have been recommending people upload their lists to GitHub and then they can just be added as any other list would.

You could also make a custom script that would download and unzip the file…but that might be more work then you want to get into.


#3

That might even be dangerous. 000 is no real HTTP response code, but how could it? When you “download” using file:// there is actually no HTTP server that could respond with anything. I would suggest to put it in some place where you can use Pi-hole’s webserver?

Having said that, the response code 000 will always be there, even if the file:// path is wrong (i.e. there is no file here) or the permissions are wrong and curl cannot read its contents. So I cannot simply go ahead and accept the condition 000 + file:// as Success.


#4

@jacob.salmela I’ve already made a script which downloads and extracts the file, but that’s how I end up with the same problem. Or did I
The upload to Github is a workaround I have not thought of, I will keep this in the back of my head and do this if I can’t find a solution.

Yes,@DL6ER you’re right, that’s true. I have not thought of this.
But what if we simply don’t use curl if we detect the file:// part? We could simply use cp or cat, which provide error codes if the file is not found or we don’t have read permisions.


#5

But what if we recommend to use lighttpd instead? :blush:

The next update will most likely be v3.0 (including the FTL engine) and hence will not come within a week or two, so you’d have anyhow to wait quite some time until you’d get the new feature. However, the lighttpd approach is already available today and is also reliable.


#6

Ah yes, this is also an idea. I will try to figure out how that works. Thanks!


#7

Just go into the web directory and create the file there, e.g.

root@raspberrypi:/var/www/html# echo "abc" > mylist.txt

then you can download it

pi@raspberrypi:~ $ curl http://127.0.0.1/mylist.txt
abc

#8

I completely forgot about this feature request. However, I worked on this issue independently today.


#9

Thanks! I totally forgot about this too, because I got rid of my local block list and used a “normal” one instead.
While the workaround with lighttpd worked fine, I appreciate your effort that went into implementing this!


#10

This has been implemented and will be released with Pi-hole v3.2.


closed #11