FilterLists

Seems to have parsed them correctly then, every entry is:

192.168.1.105 somedomain.com

Good stuff because a lot of the lists I added were not in hosts file format at all

We do some sanitizing of domains as they are added and try to add domains, but with non-formatted lists you may not be getting all of the domains included.

That's fine, as long as it only adds entries it can use, and not just blindly adding everything in the list in the format the list is in

I have a script that should parse all the lists on that filterlists site. It should run at midnight-30 EST daily. It will then update to the lists I have on GitHub. These will need testing. I also made it so that lists can be added super easy.

https://github.com/deathbybandaid/piholeparser

So we just run your code written in the instructions on Pi.Hole to add all the parsed lists?

Within the repo is a directory titled 'parsed' I should have been more clear. I already parsed the lists.

Just checked and my automatic script ran without issue!

I parsed all the lists separately on purpose.

Ah I see, so we have to add each of those lists manually to the lists.lst with Nano?

Add the parsed list directly to pihole either with the webui, or

sudo nano /etc/pihole/adlists.list

Will adding just https://github.com/deathbybandaid/piholeparser/tree/master/parsed URL to adlists.list add all the parsed lists or we still need to add them one by one?

I need to update my README.md, I just ran out of time last night.

You won't need the tool to process the lists,,, just the lists.

I did all the heavy lifting.

One

By

One

I made a choice to not merge all of the lists into one. These are simply the original list maintainers work, but filtered/parsed.

That and the way it is setup, I can easily add more lists to parse.

Well that was no fun, but my contribution is this pastebin of all the URLs in that parsed list you created

One by one (My eyes hurt now lol)

Thanks, I've been on mobile and didn't have the chance.

https://github.com/deathbybandaid/piholeparser/blob/master/All%20the%20lists.txt

There may be more lists to come.

Added all those lists to my pihole but it only seems to have added a few more more blocks, less than 50

Can you post the URLs you're using to get 1.6m blocked?

This is what I'm using,,,, everything I use is within the piadvanced script,,,, consult my modules regarding installing the cronjobs and adding the lists to the files. (I'll be on mobile for most of the day.)

The Wally3k Adlists

https://github.com/deathbybandaid/wally3k-adlists.list

Some Youtube adblocking

https://github.com/HenningVanRaumle/pihole-ytadblock

More Youtube Adblocking

(nano the files to change directory if not in /etc/piadvanced)

https://github.com/deathbybandaid/piadvanced/tree/master/piholetweaks/youtubeadblock

Adguard

https://github.com/deathbybandaid/piadvanced/blob/master/piholetweaks/adguard.sh

This is a php parser, put it in your /var/www/html/admin/ folder

https://github.com/deathbybandaid/piadvanced/blob/master/piholetweaks/parser.php

I also use this to remove lists once a week, to re-download them. Some Lists don't seem to correctly distinguish if they have changed or not,,, so this enables a fresh download.

https://github.com/deathbybandaid/piadvanced/blob/master/piholetweaks/piholefreshlists.sh

1 Like

Well I've removed just about all the lists from the OP / parsed github, seems like every day I was having to whitelist yet another normal website, gmail, tech forums, kickstarter, yahoo mail (Don't use it, but discovered it was blocked) and other seemingly harmless websites such as Amazon

Got sick of having to whitelist constantly, and the gf complaining yet another X app stopped working, or wouldn't let her log in, or a site wasn't working

Default set of lists with a couple of others and back to 108k blocked domains now

https://github.com/chadmayfield/pihole-blocklists

This one really increased the amount I block!

There's an occasional domain that I have to whitelist,, but nothing has been super inconvenient for my wife,,, so it must be alright!

2 Likes

Holy moly, didn't realise there were quite as many pR0n sites out there lol

There's only me and the gf here so no worries about kids accessing things they shouldn't, ads are the main thing I want rid of, and facebook, but she uses that so I can't, or it would be wildcard blocked

if that's true we should really be okay haha

Don't know what's going on with mine, you have over 3m blocked and no issues other than the odd domain, I have added again only up to 700k and it's blocking things like mail.live.com, and these lists I am adding are malware domain lists recommended by people already using them (Can't find the site atm)

Why would live.com be on any list at all? Some very paranoid people tinfoiling about MS Win 10 telemetry spying to block an email domain and add it to a malware filter list