Blocklist approx 1.9 million domains includes my allhosts and @hagezi multi blocklist

General Customizing Pi-hole
,
1

My piholes where getting very slow on the domain page with over 900 on the list so I fixed it.

I have been using the @hagezi multi list as well for some time now.

I wrote a program in QB64 to process my lists so I can whitelist sites when my domain list grows too much.

https://smokingwheels.github.io/Pi-hole/multiedit

see screen dump

3

Hello,

you write that you use my multi list (version normal?) and there was a lot to whitelist in it. This surprises me a bit, because:

  1. the normal does not use aggressive sources.
  2. all lists were tested against 6000 websites from the toplist. It was checked whether the page loads, the page layout is correct, videos load, images are displayed, the navigation works, ...
  3. in the course of the test a whitelist was created, which is used for the generation of the lists.
  4. the feedback on false positive domains is very low.
  5. dead domains are removed regularly
  6. the lists do not contain referral domains

The current versions Light, Normal and Pro of the Multi list are actually lists that you can add and run without much whitelist orgy.

I am interested in exactly which domains these are so I can check what I missed and whitelist them if necessary.
Please provide me with the whitelist.

If you find false positive domains in lists please report them to the maintainer, then everyone will benefit.

Thanks and greetings,
Gerd - GitHub - hagezi/dns-blocklists: DNS-Blocklists: For a better internet - keep the internet clean!

4

I checked your generated multiedit list against my whitelist of falsely blocked domains (without using referral domains whitelist) and for dead domains, here is the result:

grafik
grafik870×179 17.3 KB 

585 false positive domains
121286 dead domains
1805660 domains after cleanup

in your list e.g. the following domains are blocked:

spotify.com
kino.de
msn.com
pcwelt.de
...

All incorrectly blocked domains: multiedit whitelisted - Pastebin.com

1 Like
5

Not too sure but the file is 19543404 bytes long.

I have 11 lists in my collection could be one of them but now it is all one list in my pihole.

Here my whitelist its what I'm using atm with my 11 lists.

https://smokingwheels.github.io/Pi-hole/exwhite.txt.

I have added a few to my blacklist as well your welcome to add them may cause you issues.

https://smokingwheels.github.io/Pi-hole/exblack.txt

I use them because of my yacy search engines.
See https://github.com/yacy/yacy_search_server

For other users and my own reference I used these commands on the pihole to generate the lists thus removing them from the domains page. There are 4 types.

https://discourse.pi-hole.net/t/export-and-import-of-blacklist-blacklist-wildcard-whitelist-and-whitelist-wildcard-as-4-txt-files/57179/7

Also downloadable at https://smokingwheels.github.io/Pi-hole/genlists.sh

6

Thanks for that. I do have some lists for spotify.

see https://discourse.pi-hole.net/t/spotify-ad-block-list-80-working-with-whitelist-plus-allhosts/55547

My yacy search engines have also have queried some of them as well when crawling web sites.
I have a pihole log extractor program in QB64.

https://raw.githubusercontent.com/smokingwheels/smokingwheels.github.io/master/qb64/piholelogext.bas

7

Thanks, on a quick check of your whitelist I noticed that there are a few domains on it that are not false positives but are flagged for tracking, ads and as dangerous: remvoe from exwhite - Pastebin.com

Examples:

Malicious, check https://www.virustotal.com/:

torrent-protection.com
undo.it
bottlestonightapp.com
sordidsmile.com

Tracking, number one on the net:
google-analytics.com

I recommend you remove them from your whitelist, I am not aware of these restricting functionality. For reference, they are also blocked by OISD list.

In short, I can not confirm your statement in the first post that would be much to whitelist on my multi list.

As for your blacklist, I noticed at first glance that it contains a few *.arpa domains, these have no place on a blocklist, see: .ARPA Domain

8

I use a free subdomain service from https://freedns.afraid.org/ which was blocked.

Thanks but its easier to blacklist them now its inverted kinda.
I only have regex blacklist entries on my domain list.
(.|^)myworkdayjobs.com$

yes I have been using google recently to do stuff that possibly led to that domain, by all means remove it.

Its a personal choice I did not like them showing up all the time in the logs. Yacy can cause some unexpected results.
The internet still sort works ok if you add (\.|^)arpa$ as regex blacklist.

9

A better solution in testing.
regex in whitelist (\.|^)192\.in-addr\.arpa$

regex in blacklist (\.|^)in-addr\.arpa$ regex in blacklist