Inaccurate blocked domains count

[Mausy5043]

The webpage shows this:

which is consistent with what pihole-FTL thinks but not what I find:

$ grep Gravity /var/log/pihole-FTL.log
[2017-11-18 10:58:50.854] Gravity list entries: 107094
$ wc -l /etc/pihole/gravity.list
133345 /etc/pihole/gravity.list

That's a difference of over 26000 domains!
Can someone explain that please?

token: ds3eiaaypb

[Mcat12]

Try restarting FTL:

sudo service pihole-FTL restart

[Mausy5043]

Been there, done that.
Even rebooted but to no avail.
Difference remains.

[diginc]

You may need to dig into the contents of gravity.list to see if it truly has 26000 extra unique entries in it. Are you running a custom gravity list or custom external lists? when you just run gravity.sh manually how many domains does it list as unique?

My guesses on the cause: comments and bad domain addresses wouldn't be counted in the machine total where as they would in a word/line count command. I believe gravity.sh will remove all comments from lists by default so unless you run a completely custom gravity.list it is probably not that. If you have bad domains in your lists DNSMasq should be throwing errors (check your pihoe log) as it reads your gravity.list.

[deathbybandaid]

does the amount change if you disable one of the lists?

[Mausy5043]

I run
cat /etc/pihole/gravity.list | sort | uniq > /tmp/tmp; mv /tmp/tmp /etc/pihole/gravity.list
after I've added my own list.

[Mausy5043]

Just to give some background:
I have several years of experience with running dnsmasq. I see no errors in it's logging.
I add my own list to the existing gravity.list and screen it thoroughly, removing duplicates and whitelisted sites.
I could post my sources if that would be of any help.

From the source of pihole-FTL I gather that it "just" counts lines similar to what wc -l does.
If that is a wrong assumption I'd be glad to be educated.

[deathbybandaid]

Okay, going to try and rule some stuff out. Bear with me.

First thing I'd like you to try is:

mkdir /var/www/html/lists

cp /path/to/your/list.txt /var/www/html/lists/customlist.txt

THEN via the webui,

add http://YOURPIHOLEIPADDRESS/lists/customlist.txt

then update your blocklists, either from the webui, or by running pihole -g

Let me know if that changes your count.

[Mausy5043]

After adding the list via the WebUI, it said that it was "Updating..."
Relevant part of logging:

::: Aggregating list of domains... done!
::: Formatting list of domains to remove comments.... done!
::: 263679 domains being pulled in by gravity...
::: Removing duplicate domains.... done!
::: 133374 unique domains trapped in the event horizon.

Result:

$ wc -l /etc/pihole/gravity.list
133375 /etc/pihole/gravity.list

Now it's updated!

This begs the question, whether pihole-FTL keeps a record of the count somewhere and doesn't update it when it gets restarted?

[deathbybandaid]

Officially, we don't support modifying gravity.list directly.

We are looking into why the number didn't update, because in reality it should have.

However, the option I gave you is a functional solution until we apply this PR to add the ability to add local files directly.

Add support for local blocking lists

Hope that helps,