I concatenated every BlockList I could find

smokingwheels · October 19, 2017, 3:59am

I concatenated every BlockList I could find then transforming the lists to start with 0.0.0.0 and 127.0.0.1 depending on source.
It took Over 50 Man hours to create and modify this list with the bulk of the lists provided by @WaLLy3K

https://smokingwheels.github.io/Pi-hole/allhosts
Includes InBrowser Mining and sites that run them. You may have to WhiteList a few sites.

I am running on 2 Pi-Holes currently for the main use of my YaCy Search Servers.
I am running 1 on a Raspbian 3B it consumes around 2 Watts on the power bill it cheaper than a cloud and more convenient.

The Actual number on a clean Pi-Hole is 978022 Domains on BlockList I may update with a few more like the pic after testing.

Updated and bigger list.
https://ncloud.zaclys.com/index.php/s/MPYSjLkb8cE8gPN/download

WaLLy3K · October 19, 2017, 4:12am

When hosting stuff like blocklists on GitHub, please make sure you do not use raw.githubusercontent.com links. Since these links don't provide the last-modified header, they will get re-downloaded every single time pihole -g is run (or each week automatically) even if there are no new changes. That's a waste of time and bandwidth

Since you've created a GitHub Pages Website, you can use the following URL which ensures only updates are downloaded: https://smokingwheels.github.io/Pi-hole/allhosts

smokingwheels · October 19, 2017, 4:19am

Ok done I have changed it, Thank you.

There must be some sort of delay in github because I tried it and got a 404 error soon after the first push.

I will make a note of that for the future.

There are three lists now I separated my YaCy Search Engine from the main list.
I have added another list and room for 1 more.

Right click Copy Link Location and paste into your Pi-Hole at /admin/settings.php?tab=blocklists then save.
When complete Save and Update.

Added 2 Lists from QB64 Pihole log extractor.
https://smokingwheels.github.io/qb64/piholelogext.bas

If you want to add something to my list please contact me Via Twitter @smokingwheels or here.

frankrpi3 · October 20, 2017, 9:11am

Hi,

as I am using several of your lists from The Big Blocklist Collection:
There are also some linking to raw.githubusercontent.com...
So can we use them, or not?

WaLLy3K · October 20, 2017, 10:33am

Of course you can, but it's worth keeping in mind that they will always be re-downloaded even if there are no new changes

Not really an issue with small lists, but can be problematic when downloading lists over a couple of MB.

herrwusel · October 22, 2017, 12:53pm

@smokingwheels: Thanks for this awesome list (and @WaLLy3K, too of course).

smokingwheels · October 22, 2017, 1:34pm

Cool thanks @herrwusel
You may have some WhiteListing to do.
Do you run it on a Pi?

herrwusel · October 23, 2017, 5:35pm

Yes, it runs on a rp2 right now

smokingwheels · October 23, 2017, 6:15pm

Cool thats good to know.
Is it Raspbian pixel or headless?

smokingwheels · October 24, 2017, 4:49pm

My block lists are growing and the access to to micro SD card is getting a little higher than I am happy with, I might have to migrate to a HDD.
https://www.maketecheasier.com/boot-up-raspberry-pi-3-external-hard-disk/

The tool to check disk access stats is iotop.

sudo apt-get iotop

Set interval 10 Seconds.
sudo iotop -d 10

herrwusel · October 26, 2017, 8:15am

It's raspbian headless. I had to do some whitelisting for Youtube to play videos but nothing else so far

smokingwheels · October 26, 2017, 8:41am

Thank you for the feed back.

Could you share just the URL's to do with Youtube?
It maybe a country thing.

herrwusel · October 26, 2017, 5:44pm

I should have been more specific. Only on Android devices youtube videos didn't work.
I whitelisted:

android.clients.google.com
clients1.google.com

smokingwheels · October 26, 2017, 5:57pm

Thank you.
I have unlisted them.

smokingwheels · October 29, 2017, 4:02am

I have released My Personal Pi-Hole log extractor for you to have a look at your own logs.

https://smokingwheels.github.io/qb64/piholelogext.bas

Some of the domains I have found are added in searchengine1, released as experimental at this stage.

https://smokingwheels.github.io/Pi-hole/searchengine1

Happy Hunting.

smokingwheels · November 1, 2017, 1:07pm

Does anyone have a public/common WhiteList they would to share?
I am building a second list.

smokingwheels · November 2, 2017, 10:24am

I have an the log file error rate down a bit on the 2nd lists about 0.07% of domains are have Bad Names.
I have split my list because it is too big to have in one on Github.com
I have unlisted ~280 domains so I would welcome any feedback on that.
I have not tried to load it on my RaspberryPi yet but I have it running on an old laptop.
The Total content of the lists is around 140 mb's.

https://smokingwheels.github.io/Pi-hole/0nov17
https://smokingwheels.github.io/Pi-hole/1nov17
https://smokingwheels.github.io/Pi-hole/2nov17
https://smokingwheels.github.io/Pi-hole/3nov17
https://smokingwheels.github.io/Pi-hole/4nov17
https://smokingwheels.github.io/Pi-hole/5nov17
https://smokingwheels.github.io/Pi-hole/6nov17
https://smokingwheels.github.io/Pi-hole/7nov17
https://smokingwheels.github.io/Pi-hole/8nov17
https://smokingwheels.github.io/Pi-hole/9nov17
https://smokingwheels.github.io/Pi-hole/10nov17
https://smokingwheels.github.io/Pi-hole/11nov17
https://smokingwheels.github.io/Pi-hole/12nov17
https://smokingwheels.github.io/Pi-hole/13nov17
https://smokingwheels.github.io/Pi-hole/14nov17
https://smokingwheels.github.io/Pi-hole/15nov17
https://smokingwheels.github.io/Pi-hole/16nov17
https://smokingwheels.github.io/Pi-hole/17nov17
https://smokingwheels.github.io/Pi-hole/18nov17
https://smokingwheels.github.io/Pi-hole/19nov17
https://smokingwheels.github.io/Pi-hole/20nov17
https://smokingwheels.github.io/Pi-hole/21nov17
https://smokingwheels.github.io/Pi-hole/22nov17

This list has been removed. But if it grows again it will come back. Errors have been reduced.
https://smokingwheels.github.io/Pi-hole/23nov17

These lists have strange symbols on, I am not sure if the Pi-hole will accept them. 
https://smokingwheels.github.io/Pi-hole/fromlog
https://smokingwheels.github.io/Pi-hole/fromlogtosh

HellboyPi · November 2, 2017, 9:35pm

smokingwheels · November 3, 2017, 12:41am

Thank you for that.
That is one I use.
Do you know how often it gets an update in past history?

HellboyPi · November 3, 2017, 1:23am

Pretty often. You can view the update history of every list on github. AnudeepND whitelist example.