Dashboard Graphs don't match

sawsanders · February 11, 2024, 3:16pm

Every few days I see a situation where the totals of the client activity and total queries graph doesn't match; sometimes it's off by a thousand queries. The client activity is always higher than total queries when this happens.

I believe the totals in each column should match, right??

Debug log: https://tricorder.pi-hole.net/gX7IA5Y1/

sawsanders · February 11, 2024, 4:38pm

I ran pihole -up to get the latest v6 updates and the graph is back to normal. I think it must be a rendering problem that develops over time, no issue with the database.

Next time it happens, I'll just try restarting FTL to see if it fixes the problem.

rdwebdesign · February 11, 2024, 7:20pm

The sum of Total queries and Client activity should match.

If you see this mismatch again, you can check the number of queries on each graphic positioning the mouse pointer over one bar to see the details on the legend.
The sum of items in both graphics should be the same.

I really don't know why the javascript plugin calculated different maximum values in this specific case.

From this point, this is just a guess:

Maybe the issue is related to the way the maximum is calculated for stacked bars graphics (inside the javascript plugin code).
The first graphic only has 3 bars. The second one has at least 9 bars in your case. Maybe this caused a rounding difference, resulting in different presentations. (As I said, this is just a guess.)

DL6ER · February 11, 2024, 7:35pm

What makes things slow is what @sawsanders has already observed:

Please try

pihole cehckout ftl fix/overTimeGraphs
pihole checkout web fix/overTimeGraphs

in about 20-30 minutes from now (to give CI some time to build the new binaries) edit: done and check if you see the same habit coming up again.

I changed two things in these branches:

Cache-optimization was not correctly added to the total query counts, making them potentially smaller. Cache optimizer can contribute up to 20% of your queries so this is indeed relevant any may explain the difference you were seeing.
Disentangle queries by status better. This adds a fourth "other" category (blue color) in the main graph.

DL6ER · February 11, 2024, 7:40pm

github.com/pi-hole/FTL

Tweak API history endpoint and fix cached counting

pi-hole:development-v6 ← pi-hole:fix/overTimeGraphs

opened 07:39PM - 11 Feb 24 UTC

DL6ER

+15 -13

# What does this implement/fix? Include forwarded queries in `/api/history` a…nd count stale-cache queries as cached queries --- **Related issue or feature (if applicable):** https://discourse.pi-hole.net/t/dashboard-graphs-dont-match/68263 **Pull request in [docs](https://github.com/pi-hole/docs) with documentation (if applicable):** N/A --- **By submitting this pull request, I confirm the following:** 1. I have read and understood the [contributors guide](https://docs.pi-hole.net/guides/github/contributing/), as well as this entire template. I understand which branch to base my commits and Pull Requests against. 2. I have commented my proposed changes within the code. 3. I am willing to help maintain this change if there are issues with it later. 4. It is compatible with the [EUPL 1.2 license](https://opensource.org/licenses/EUPL-1.1) 5. I have squashed any insignificant commits. ([`git rebase`](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html)) ## Checklist: - [x] The code change is tested and works locally. - [x] I based my code and PRs against the repositories `developmental` branch. - [x] I [signed off](https://docs.pi-hole.net/guides/github/how-to-signoff/) all commits. Pi-hole enforces the [DCO](https://docs.pi-hole.net/guides/github/dco/) for all contributions - [x] I [signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits) all my commits. Pi-hole requires signatures to verify authorship - [x] I have read the above and my PR is ready for review.

github.com/pi-hole/web

Tweak main activity graph

pi-hole:development-v6 ← pi-hole:fix/overTimeGraphs

opened 07:38PM - 11 Feb 24 UTC

DL6ER

+43 -21

# What does this implement/fix? The permitted column in the main graph was ac…tually the sum of forwarded and other queries. As cache is arguably somehow permitted, too, make the difference between them clear. This depends on FTL (same branch name) to function properly. --- **Related issue or feature (if applicable):** https://discourse.pi-hole.net/t/dashboard-graphs-dont-match/68263 **Pull request in [docs](https://github.com/pi-hole/docs) with documentation (if applicable):** N/A --- **By submitting this pull request, I confirm the following:** 1. I have read and understood the [contributors guide](https://docs.pi-hole.net/guides/github/contributing/), as well as this entire template. I understand which branch to base my commits and Pull Requests against. 2. I have commented my proposed changes within the code. 3. I am willing to help maintain this change if there are issues with it later. 4. It is compatible with the [EUPL 1.2 license](https://opensource.org/licenses/EUPL-1.1) 5. I have squashed any insignificant commits. ([`git rebase`](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html)) ## Checklist: - [x] The code change is tested and works locally. - [x] I based my code and PRs against the repositories `developmental` branch. - [x] I [signed off](https://docs.pi-hole.net/guides/github/how-to-signoff/) all commits. Pi-hole enforces the [DCO](https://docs.pi-hole.net/guides/github/dco/) for all contributions - [x] I [signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits) all my commits. Pi-hole requires signatures to verify authorship - [x] I have read the above and my PR is ready for review.

sawsanders · February 11, 2024, 8:17pm

Ok, I've switched branches and will see if the problem is corrected.

Thanks for such a lightning fast fix to try!

sawsanders · February 11, 2024, 9:24pm

Just happened again. Both safari on desktop, safari on iPad, and chrome on desktop show same thing.

Edit to add debug:
https://tricorder.pi-hole.net/14HFA6nx/

sawsanders · February 11, 2024, 9:39pm

I ran pihole restartdns and the graphs went back to normal.

It's the "other clients' category going crazy...

and the percentages in the tool tip don't add up to 100 in the total queries graph.

DL6ER · February 12, 2024, 1:59pm

Are you using aliasclients?

Please also experiment with setting the config option webserver.api.maxClients to something very large, e.g., 100.

sawsanders · February 12, 2024, 2:35pm

Not on purpose. I see several clients have IP6 in addition to IP4 addresses in the network table, although I don't have any IP6 setup on my router nor do I have IP6 network access to the internet.

Done.

DL6ER · February 12, 2024, 2:59pm

It seems I am not able to reproduce this any longer myself.

I pushed an update to the aforementioned branch, maybe you could try with pihole -up when you find that increasing webserver.api.maxClients didn't help. This contains only some small code simplification I have made after starring at the code the entire way, waiting for the issue to re-appear (which it never did...).

sawsanders · February 12, 2024, 10:27pm

It looks like it’s double counting some clients. I can see which clients when I set webserver.api.maxClients to 100 from 10.

Could this be anyway related to client recycling? I also feel like this has been going on since Pi-hole started limiting max clients on the over time graphs.

DL6ER · February 13, 2024, 5:26am

Sure, I was thinking the same yesterday but it's totally unclear to me how duplicates could arise as recycling overwrites clients data (including counts) with zeros.

Questions:

We see here that the duplicates have the same hostname. Is it possible that they - somehow - get multiple IP addresses over the day?
Could you check the raw JSON response for api/history/clients? It should contain this information.
Is this already after you updated the branch or before?

DL6ER:

I pushed an update to the aforementioned branch, maybe you could try with pihole -up

sawsanders · February 13, 2024, 5:50am

It's possible but not very likely IMHO. My router uses dnsmasq to assign ip addresses so the ip's have been very stable every time I've checked. Also, a simple pihole restartdns fixes the graph for a several hours.

EDIT: Also, only see one entry in Pi-hole's network table for my the devices that show up multiple times in the graphs.

Yes, if you can point me in the right direction to do so. I assume that's an API call?

That's before. I have since updated and it's starting to happen again with a different client. It's been about 6 hours since FTL restarted.

DL6ER · February 13, 2024, 5:58am

Sure, this is the screen we want:

I screenshotted Chrome as my Firefox is German but the procedure is ultimately the same:

Open the Dev Tools using F12
Wherever the pan opens, select Network
Enter "clients" in the search field and reload the page to trigger the API call (F5)
Click the newly appearing clients item on the left side of the Dev Tools
Click on Response in the new tab that is opening

Scrolling down to the very bottom reveals the individual clients, e.g.,

    "clients": [
        {
            "name": "localhost.lan",
            "ip": "127.0.0.1",
            "total": 20977
        },
        {
            "name": "legion-5.fritz.box",
            "ip": "192.168.2.28",
            "total": 8168
        },
        ...

Bucking_Horn · February 13, 2024, 1:46pm

Just a note:
Do you have IPv6 connectivity?
It could be quite likely that IPv6 addresses change frequently over a day. Some OSs shift their temporary IPv6 addresses as often as once every 2 hours.

sawsanders · February 13, 2024, 2:48pm

Hmm. When I followed the Wireguard tutorial, it added an ipv6 interface to my RPI that runs Pi-hole. Many of my devices in the Pi-hole's network table show ipv4 and ipv6 addresses. I suppose I do then.

EDIT: I've been watching the live query log and I don't see any requests from ipv6 addresses.

sawsanders · February 13, 2024, 2:49pm

I only see one ip address for the current client that is being duplicated.

DL6ER · February 13, 2024, 3:01pm

This is interesting, please send me the entire output via direct message.

DL6ER · February 13, 2024, 3:36pm

The two duplicates are at the very end:

		{
			"name":	"Sam-Macbook.home.arpa",
			"ip":	"192.168.10.112",
			"total":	14782
		}, {
			"name":	"Sam-Macbook.home.arpa",
			"ip":	"192.168.10.112",
			"total":	14782
		}

This all still does not make too much sense to me - even more as this list is meant to be monotonically decreasing concerning total client count. I pushed another update that should be ready in about 30 minutes. The additional output I need will be right there in the response.