After update to pi-hole, we get:
[i] Number of gravity domains: 1020900 (812955 unique domains).
It would be nice to see the delta from the previous update, for example,
"[i] Number of gravity domains: 1020900 (812955 unique domains +2000)"
or
[i] Number of gravity domains: 1020900 (812955 unique domains) -2000)"
The gravity update happens on a regular schedule every Sunday night without any terminal output. And even if there would be logging (or by manually triggering the update) - what relevant information do you get from the delta alone?
We already keep the old database after each update, you could get the delta (and much more fine-grained information) if you compare the old and new databases. I think @jpgpi250 has some very advanced database queries/scripts he could link too.
It’s simply nice information on a manual run of pihole -g. It’s as relevant as showing the number of unique domains, even more so as there’s no context to that number. A delta at least shows how many have been added.
Yeah, I could query the old db, but if the script already knows the name of the previous db and already counts the number of domains, seems a simple matter to add this little bit of icing. If not, go ahead and close this.
I'm kind of interested in this. No need to close it in my view.
I know sometimes list maintainers make mistakes and remove or add a huge number of domains. That ends up with you either over blocking or under blocking what you expect. And you wouldn't pick up the corrected list until the next cron job firing.
A lot of the 'issues' that people have with the accuracy of Pi-hole comes down to the lists, anything that could give us insight to the list health is good. We already show users if the lists have incorrectly listed lines.
What useful information would be obtained from such a number? Adlists change frequently, as expected.
Knowing that all your adlists combined have 6,352 more domains this week tells you what?
If at some point it was justified to include the number of adlists, surely it would be worthwhile to add context to it, right?
-JDrake
As I see it, an aggregate number would lose too much context already.
I thus agree with jfb and yubiuser that there would be little to gain from having that aggregate number.
Also, blocklist counts are to be expected to fluctuate, as new domains show up, older vanish, false positives are corrected etc.
Dan's consideration of sudden increases/decreases as a potential indication for blocklist maintenance errors would suggest that knowing those deltas for individual blocklists could be beneficial.
The only issue I have with this is that people may turn to us on noticing those deltas and expect us to somehow fix them, where they should address a respective blocklist maintainer instead.
I've been using the old database to be able to see the diff between last week and this week; depending on the number of lists, this can be useful info, but often, the diff is so large (many lists), it is hard to conclude something useful.
It's much more interesting to compare the diff with the existing query log (database)
Note this result is based on query database, so the impact is real.
sometimes, not often, there are some real suprises in the result, some even require immediate action...
We're not allowed to list code here, but on the dutch forum, I've listed the queries I've been using to get this result. I'm using grafana to visualize the result, since expanding the pihole web interface isn't really an option (requires redo after every update)
What are you using as the data store? There is GitHub - nin9s/elk-hole: elasticsearch, logstash and kibana configuration for pi-hole visualiziation but ES is kind of heavy.
I'm running everything pihole related on a raspberry pi 3B
I've been using loki + promtail to visualize some logs (specific lines from unbound) directly, but the overhead was to big and the results some times inacurate (after log rotation), so I removed this.
I'm not really interested in graphs, no such dashboards... In order to visualize the data in the old database and data in gravity, I use this plugin:
# https://grafana.com/grafana/plugins/frser-sqlite-datasource/
sudo grafana-cli plugins install frser-sqlite-datasource
The plugin has some limitations, not all valid sql statements work, for example you cannot use two databases (attach). That is why I prepare the results in the old database, grafana with the plugin only shows the results from the tables the way I like to analyze them.
the diff from today (gravity ran last night), nothing to show after this run...:
some other grafana dashboards, using this plugin:
edit
you'll recognise some of the domains (result of testFTL script I run after an update / fresh install to verify FTL is doing everyting it should - you are using PowerDNS to do this). The cc TLD is considered bad by suricata, thus dropped (on the firewall), result -> SERVFAIL
details for a specific domain? again, click on the link -> other dashboard
/edit
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.
