Must say I'm rather surprised to not find this feature in such an advanced project. Already been using the feature with diversion.ch (who offered it from the get-go). In fact, I'd say this is a Nr 1 requirement and its lack is quite off-putting. I'll have to go back to pfsense's filter or diversion if this doesn't seem to ever be implemented. And I would code it myself if I'd have the time right now, it's really not that hard to do.
For now, here's a whitelist I've been maintaining for diversion for ages, it's a concatenated bunch of lists plus my family's own, and some sorting/cleaning of it all; https://jult.net/white.txt
I did not write anything about writing pi-hole in perl. Again;
pihole offers a box in a web-ui where one can enter hostnames to be whitelisted, it works when I paste hundreds of them in there and submit them for whitelisting. Those hundreds of names are copy pasted from a txt file I create and maintain myself, because I use pfsense and diversion for more than one machine and network I administrate, and I'd rather have the whitelisting to be centralized for me (and others like me who trust my judgement, which, by the way, many seem to do, since the list is being grabbed a lot).
Since one can enter the list through copy pasting into a web-UI, it seems rather trivial to generate the sqlite from a given URL. In fact, the code is almost done; GitHub - dMopp/pihole5-adlist-update-cron: A pihole5 compatible cronjob to fetch Adlist(s) from URL and import them to the new gravity.db
Sharing and importing other user's whitelists does only treat the symptoms and is not a cure: you got to many or/and badly maintained adlists.
Your list contains almost 1000 entries, including wikipedia.org. If I would have to whitelist wikipeda, I would start wondering if I use the wrong adlists....
Sharing whitelist has another drawback: I have to check the whole whitelist for entries I would like to keep blocked. For instance, your whitelist does contain facebook.com. If I would want that to be blocked, I could not use your list, because whitelist takes precedence for blacklist - so there would be no way for me to block FB anymore. But if I have to go through the list anyway, I could also whitelist the few entries I really need myself - giving me also the opportunity to dis/enable it for individual clients.
It's not that easy if it's done right: You would need a new UI section, modify the downloading and paring algorithm to import into whitelist (a new whole whitelist table as for gravity domains right now? Or into the domain table?) and a new table for storing the "imported whitelist".
What the code you linked does is rather trivial, that's right: only adding the URL of adlists into the database. But this is only the initial step - lot of work has to be done for paring, storing, maintaining.
I use this whitelist myself, because it works for me and has worked for me for quite some time, the only reason I bumped into this thread is because I can't put down the URL so that pihole uses it for its whitelisting. I could care less about your reasons not to 'trust' me. I'm not "tracking the users downloading my whitelist", I run a self-hosted nginx webserver, it has an access log. Sometimes when I work on the server, I need to check what's going on, and then I notice lots of pulls from that one list. I have never asked anyone anywhere to do so. They are free to do it, and I'm a member in some large groups (even some on facebook) where, rightfully, they all know where I'm coming from and probably most of the grabs come from that; There's so much info on me personally to be found online, the mere blog on jult.net alone speaks a thousand words. I've been online since the fidonet and usenet days, so I know quite well where to draw the line between on and offline, to be honest. It's really not that hard to trust people if you're savvy enough to sift through the noise, especially not when they have a proven track-record for decades in a row. Just like how I fully trust Raymond Hill in his judgement regarding privacy and ads. I have a similar manifesto.
Pi-hole is a great project, the web-gui is amazing, but it lacks in the whitelisting sources options and I really don't know why that's not just in there, just like with the blacklists. Why trust some obscure source to block access, yet distrust those allowing access? Pi-hole as a cleaner/security-feature is always a 'best effort' concept, you really shouldn't be so paranoid about whitelisting.
Blacklist goes rogue and the worst that happens is you have extra domains blocked. Whitelist goes rogue and you get Google/Facebook/Microsoft/BigBadOfTheDay allowed to track you and you'll never know since it's not ads blocked but privacy violation.
Just adds to my list of reasons not to trust you.
If this was your attempt to persuade me that this is a needed feature then you failed horribly.
That's the exact reason why I wouldn't want to see this. Yes, we can assume this will not happen, but this is like saying "Microsoft* makes bug-free software".
*) replace by any other large software company
I see users adding 30 block lists and compete in a way like "I have X blocked domains, who has more?". These guys would add a whitelist which claims to "allow Windows updates" but, in fact, it contains a lot more. Many many users are beginners and they cannot oversee all the consequences of what they are doing. And when I cay they cannot please don't treat this as I'd say they are stupid. They are just lacking the experience to tear things apart.
First, I agree with the sentiment that trusting someone else's whitelist is not the best idea. It could leave holes in your system for unwanted things. I fully agree with that.
That said, I would still like the option to use whitelists, preferably with the option to assign them to groups. For me, I'd like to create my own lists to use without cluttering up the web gui. For example, I have a computer running torrent software. Several torrent trackers are included in blacklists. I'd like to create a list of the domains that I want to permit and allow them only for the 'Torrent' group so they are still blocked on the other computers. I currently have the GUI whitelist page cluttered with them. It would be nice to just have a simple 'torrentwl.txt' file that has all of them in one place, easily managed, and not cluttering the GUI.
Instead of adding and then removing, how about distilling your own adlists-without-torrent.txt that doesnt have said domains to even begin with? This new blacklist can then be the only source for your torrent device group.
Its very easy to import a list of domains contained in a file with the pihole command and xargs:
Usage: pihole -w [options] <domain> <domain2 ...>
Example: 'pihole -w site.com', or 'pihole -w site1.com site2.com'
Whitelist one or more domains
Options:
-d, --delmode Remove domain(s) from the whitelist
-nr, --noreload Update whitelist without reloading the DNS server
-q, --quiet Make output less verbose
-h, --help Show this help dialog
-l, --list Display all your whitelistlisted domains
--nuke Removes all entries in a list
--comment "text" Add a comment to the domain. If adding multiple domains the same comment will be used for all
pi@ph5b:~ $ xargs pihole -w < whitelist.domains.list
[i] Adding domain.1 to the whitelist...
[i] Adding domain.2 to the whitelist...
[i] Adding domain.3 to the whitelist...
[✓] Reloading DNS lists
pi@ph5b:~ $ xargs pihole -w -d < whitelist.domains.list
[i] Removing domain.1 from the whitelist...
[i] Removing domain.2 from the whitelist...
[i] Removing domain.3 from the whitelist...
[✓] Reloading DNS lists
Or from web:
pi@ph5b:~ $ curl -sSL http://localhost/whitelist.domains.list | xargs pihole -w
[i] Adding domain.1 to the whitelist...
[i] Adding domain.2 to the whitelist...
[i] Adding domain.3 to the whitelist...
[✓] Reloading DNS lists
I really think that allowing subscriptions to both exact whitelists and regex whitelists would be incredibly useful. I've just posted my use cases on another, related issue and would rather not repeat myself verbatim here.
Of course there are risks from using publicly maintained lists from people / companies with agendas (paid to whitelist ads etc), or those lists which are compromised (this exists for exact blacklists too - simply delete the list content), but what about those of us who simply wish to maintain their own lists on Github for personal use who don't want to mess with SSH, Python scripts or slow interfaces every time they want to just add an item (especially when coupled with Group management for 100s of individual rules)?
Please, please consider this. From looking at the system that currently exists (which is amazing!) I don't think that supporting different kinds of lists (in addition to individual items) would be beyond the realms of possibility. You could even put the feature behind an advanced option to prevent 99% of users ever encountering it...
This feature would be very helpful, as we can maintain the whitelist somewhere that the users can submit pull request/merge request to update it, yet those users inside the network won't need the permission/password of Pi-Hole.
That'll work for some SOHO network environment, or those personal projects that using Pi-Hole to provide AD-blocking DNS service.
Also, it's more maintainable, if you have more than one Pi-Hole instance, that you may setup Pi-Hole at different places, or even just multi instance at one place for high availability.
If you have multiple sites, or multiple piholes you need to keep in sync, having the ability to subscribe to a whitelist makes this much easier as you dont have to add entries across devices manually. Really wish Pihole had this. Been wishing for this for a long time. Would be super helpful.
