Don't run pihole updatechecker every 10 minutes on branch master

I found the following in /etc/cron.d/pihole (pihole version 3.2)

# Pi-hole: Grab remote version and branch every 10 minutes
*/10 *  * * *   root    PATH="$PATH:/usr/local/bin/" pihole updatechecker

this will execute /opt/pihole/updatecheck.sh EVERY 10 MINUTES

why so frequently, can we change it to once a day without impact on pihole?

Here is the answer:

The reason for this frequency is not to check for updates very often, but to detect the branch the user is currently on (the determination of the current hash is also outsourced into cron as well)

Sorry, but I don't get it.
As a result of this, api.github.com is now the top domain, 811 hits since (fresh) install.

As a result of this cron job, a file /etc/pihole/GitHubVersions is created, content:

v3.2 v3.2 v2.12

I also found a new file /etc/pihole/localversions, content

v3.2-0-g5ba4135 v3.2-0-gf74cd82 vDev-437af07

and /etc/pihole/localbranches, content

master master XXX

I understand you need to check the current version versus the available versions to allow the web interface to show reliable update info and pihole -up to function correctly, but fail to see why this info needs to be updated every 10 minutes.

Even if somebody forces pihole -up without the correct information available in these files, wouldn't it be easier to just retrieve the correct info as a part of pihole -up.

Sorry for not letting this go, but, if some device is going to the internet every 10 minutes, I want to know why this is necessary.

I think more than 90% of the users are on the Master branch. In my opinion there is no need to check every 10 minutes, way too often. An update check max. twice a day is enough.

Some valid points. @DL6ER what do you think?

My suggestion:

User is on Master branch: check every 24 hours
User is on Dev branch: check every <Pi-hole maintainer/developer decision> minutes/hours

1 Like

I shifted this to the Feature Request section. I may just add this at some point when I find the time.

# */10 * * * * root PATH="$PATH:/usr/local/bin/" pihole updatechecker

Does it for me - also wondering about this new behaviour.
People are spending much time in config their pi-hole to block this kind of tracking, and now pi-hole itsself is (more or less useless) checking for updates every 10 mins? :roll_eyes:

nevertheless. very good job!

Sorry, there is nothing to vote. The current behavior was a wrong decision and doesn't make any sense for nearly all users. Please show me other software that checks for updates every 10 minutes.

So far the manual workaround is to change the interval in the cron file to a reasonable value.

I did never say that there is something to vote. However, this is not a bug (the update checker is working flawlessly nevertheless) but rather a feature request so this belong to this category.

Also, although I didn't say this explicitly, I will change this behavior rather sooner than later.

Not sure what you mean by "this kind of tracking", as there is no tracking involved. There are very good reasons why we query the versions from Github and process them in a rather complicated way. It would be much easier to just host files with pre-parsed versions on our servers, but then you could argue that we could potentially be tracking you. However, how it is done right now we get the versions from Github's API and there is no way for us to get any raw data from Github about who did query what when.

It wasn't a wrong decision at all. Previously, the web UI checked for updated on every page loading, i.e. that may have been dozens of times in a single minute. I don't see how the current behavior is worse. I already agreed on that it is of no real use for the arbitrary user and I also already set that I'll work on this...

You don't even know how most of the software is checking for updates. People use closed-source software all of the time and they may contact their mother ship only on start-up or even more frequently than every 10 minutes, who knows? This includes also pupular products like Android etc where updates are "suddenly" mentioned - do you know in which interval the database is checked for updates? I don't.

1 Like

@DL6ER thanks for your response and explanations. Thanks to all developers for your time and great work on this project, keep going :+1: Pi-hole is brilliant.

1 Like

Please review

1 Like

I'm not sure I follow. Looking at my long term data, I had 92 queries for api.github.com total between Nov 15 and Dec 6. Where was the web UI checking for updates before this?

The DNS lookups would be cached both at the client and in Pi-hole. So you don't show a query for each time a client requests an asset, only the first time when it requests the IP. You will sometimes see a cached response in the logs, and that's when the Pi-hole is responding with an IP from it's cache. When the client has cached the IP address, there is no query to the Pi-hole and thus there is no record of it happening.

Edit: Clarity of point.

But right now Pi-Hole is forwarding one query and responding to two from its cache every 10 minutes. Do the same rules not apply to both the old and the new update system? My Pi-Hole handled 92 requests (cached or not) in a 22 day period previously, but over 400 queries yesterday. Are you saying that client cache "hid" a similar number of update checks? Were update checks previously done by the client accessing the web UI and not the Pi-Hole itself?
I understand why checking for updates on every page load (I assume this is isolated to Pi-Hole web UI loads) is not desirable, but for me the software is mostly set-and-forget so it didn't really matter.
Could it have been implemented to happen on page load but only if a certain amount of time has passed? Sort of like:

if (lastUpdateCheck <= tenMinutesAgo) {
    updateCheck();
}

And in updateCheck():

if (successful) {
    lastUpdateCheck = currentTime;
}

EDIT: I want to clarify that I am satisfied with the proposed feature change by DL6ER, I'm just trying to better understand the updatechecker.

For a lot of people, pihole is a 'set and forget' piece of software. Personally, I check the pihole web interface once a day, and, whenever something is blocked that shouldn't be (rarely happens).
I thought the code to check the version was part of the footer (scripts/pi-hole/js/footer.js). Avoiding a version check on every page load could be easily achieved by storing the versions in a cookie with a lifespan of a day.
cookie valid? -> display the stored values
cookie expired? -> retrieve latest values and update cookie

As an alternative, if you don't want to store the info on the client, use files in /etc/pihole and check the age of the files (last update check) by checking the file times.

No cron job required, no load on the system, contact github just once a day, ...

just my two cents

meanwhile, I just edited /etc/cron.d/pihole and changed the entry into:

# Pi-hole: Grab remote version and branch once a day @ 05h29
29 5 * * * root PATH="$PATH:/usr/local/bin/" pihole updatechecker

Yes, the issue was that we did it exactly as you proposed. However, querying the version delayed the page loading every certain time (and even if it was only once a day) by several seconds which became annoying over time. It was quite severe on low-end devices as the Raspberry version 1, where the first loading of the web interface, e.g. after 24 hours took over ten seconds, while afterwards everything was fine.

See my concerns above. The behavior that is implemented by my pull request does exactly that - check only once a day. However, it will shift the delays caused by querying the API and git (multiple times) to cron, i.e. doesn't affect page loading time.

This is solved in v3.2.1, which was released today.

Its not resolved since it occurs also in FTL 5.8.1.

Can you be a bit more specific about what you mean when you say "it" ?