New Blocklist over 5 Million

Try this one you will need to whitelist a few and it takes some time on a Pi 3.
Over 300 lists 480.5 mb we will see how this host goes.

https://cloud.owncube.com/s/3NYXeWWDfD69CB7/download

1 Like

over 20 million domains in the list, lots lots lots of double, bolizarsospos.com exits 91 times.
I have removed the double and the filesize shriked dramatic with 60%

1 Like

Hi smoking wheels can you reduce the size of this file by removing duplicate URLs and false blocking of websites and upload it to your hosting site pls???

Hi, I edited the file with ULM and removed the duplicates.

Here’s the link if you want to host it somewhere :slightly_smiling_face:

https://mega.nz/#!Vw5VjKgD!XmFxk2lm332Ugzjky_sl4sHwC9neVWkIljIn4cbJ2_E

1 Like

Hi thanks for your reply. I am trying to upload it now.
Can you tell me how to create systemd file? I am trying to get dns over https setup.
I am struck there in creating and enabling cloudflare service. Can you help me pls ?

Off topic, but try here:
https://scotthelme.co.uk/securing-dns-across-all-of-my-devices-with-pihole-dns-over-https-1-1-1-1/

If you don’t mind me asking, what is the purpose of blocking this many domains? Were you seeing ads or other unwanted material? Are you targeting specific things? Trying to get a big number in the red box?

less hosts4 | sort | uniq > hosts4_new

from 20.000.000 to 5.800.000 entries

$ wc -l hosts4 hosts4_new
 20376417 hosts4
  5887544 hosts4_new
 26263961 total

great :+1: but there are still almost 200.000 lines with just only an IP adres, and lots of junk.

Any way to even condense and make it more small and faster loading of this blocklist possible?

Moving repeating sub-domains to regex (Wildcard) could reduce the size. Why such a huge list and you will only have a small percentage of hits in that list.

I have a hit percentage of 34% with 106.000 domains in the list. I have ‘optimized’ regex which explains the percentage.

How did you optimize the regex in wildcard setting?

mmotti has written a complete script to do this. This should still work on the current version of Pi-hole.

mmotti GitHub optimize gravity

Still get ads on Sony Android TV and cell… along with tablet when watching YouTube. So pretty pointless. And yes it’s set up correctly.

On Youtube there are serveral threads. Youtube uses a special method to avoid Pi-hole being effective.

yes just trying to get a big number.

The server was unable to complete your request.
Cant login to source. Host