We all know pihole is only as good as the lists we use. This reddit topic indicates that pihole is doing ok with the default lists, however, some people always want to add more lists.
To check the added value of new lists, I've written a script that uses gravity.sh (part of pihole) to determine if there would be any new entries in gravity.list (the list pihole-FTL uses to block ads).
The script creates a new directory in /etc. No way this can be changed, gravity.sh will complain if this is not the base directory.
The new directory is /etc/checklist. All necessary files will be copied or created by the script.
It is possible to use tmpfs, thus saving wear and tear on your SD card. To enable this, you need to uncomment useramdrive (line 46). To remove the ramdrive, follow the instructions in the wiki
As an example, I'm running the script with a new list, this is the result:
As you can see, in my setup, there is no added value to adding this list, there are 6510 entries in the list, but it contains 0 new hosts. edit This does NOT mean this is a bad list! It only means I already have some other list(s) that contain the entries in this list. /edit
You can always check which hosts would be added to gravity.list (the active list, used pi pihole-FTL), by checking the file /etc/checklist/gravity.list
This script doesn't affect pihole in any way, commands that would affect pihole have been replaced with a dummy script. This implies DNS resolution should be functional, before running this script.
This script makes no changes to your pihole installation.
This script creates a new directory /etc/checklist, all writing is done in that directory.
If you enable the useramdrive function, changes will be made to /etc/fstab
You note that this is "Checking the added value of a new blocklist."
This checks whether the new list adds any domains to gravity. That has no bearing on "value," since the added domains may be NXDOMAIN or domains that you would never visit anyway.
Adding a domain to gravity does not necessarily equate to higher value.
The entire gravity.list contains entries that could possibly match this description. It is inherited to using lists that you haven't made or checked yourself. To overcome this problem, this would be a solution, but running that script takes hours, if not days, It's fairly pointless to run that script, unless you absolutely want to know the real value of gravity.list.
Running this script (checklist.sh) takes only a minute or so, and would give you at least an idea of how many new entries would be added.
running that script takes hours, if not days, It’s fairly pointless to run that script, unless you absolutely want to know the real value of gravity.list .
My point is that quantity does not equal value. It equals quantity.
not a bad idea to be able to check against stock list, but if you add https://v.firebog.net/hosts/Easyprivacy.txt you'll get about 4000 new entries in gravity! Any bugs still in code?
According to my screenshot, there are actually 6510 valid entries in this list.
As I tried to explain:
I have an adlists.list that contains a number of lists. When pihole -g runs on sunday (cron), a new gravity.list is created on MY system (unique, because it is created, using MY adlists.list).
You probably have a very different set of lists in your adlists.list file, or if you haven't added any, you would use the default lists, as provided during the pihole installation. When pihole -g runs on your system, a new gravity.list is created on YOUR system (unique, because it is created, using YOUR adlists.list).
The script calculates the number of entries that would be added to the, already existing /etc/pihole/gravity.liston the system it runs.
The result of the script will vary on different installations of pihole. On MY system, there wouldn't be any additional entries. On YOUR system, it appears this list might be useful (about 4000 new entries).
