Randomize time of cron's PiHole -g process for getting all files


#1

Several times I see in my log that the automatic gravity update on Sunday 1’59" gives a Status 000 (List download failed, using cached list (list not updated!)) error.
The cause for this that in a time zone all PiHoles, located in that time zone, access the webservers, mentioned in the adlist at the same time (exact the same minute !!!). And weak servers (f.i. zeustracker.abuse.ch) cannot handle more then 1000 requests for the block-file.
I have solved this by changing the time (only minutes) manually and after that no problems like this.

Is it possible that when the Pihole is updated (pihole -up), the cron task pihole gets a random time for the minutes by the PiHole -up process? Thus in every update process a new random time (only minutes) is stored in the pihole cron task. Advantage is that all PiHole have a different time for running the automatic gravity process


#2

I have your exact fame feeling about this. I also update once a week or two if possible. Could caring too much be a bad thing? :smiley:
Unfortunately I can’t vote anymore. I have voted only like less than ten times.

The good news is the the pi-hole script has curl command-switch option that will download only if the timestamp in the http header is newer than the downloaded one. Bad news is github raw files don’t have http headers and therefore get downloaded every time.


#3

This is actually an interesting feature request, and probably would not pose a problem if our userbase was small, but we are beginning to grow so I suppose it is possible. I personally haven’t heard of this happening to anyone else, but I can see it happening. It may be worth while to randomize the time–even by a few minutes–to be a respectful netizen.


#4

I discover this by writing the logging of updateGravity into a logfile and mailing this file to me.

Therefore change in Cron (in directory /etc/cron.d/pihole):
59 1 * * 7 root PATH="$PATH:/usr/local/bin/" pihole updateGravity >/var/log/updateGravity.log

I also have done this for a piHole for family including mailing.
In both cases I saw the problem: a “Status 000” error during the updateGravity process with the same web server.

I think the easiest way of implementation is: take the minutes part of the time of installation/upgrading the PiHole and put this in the cron task time instead of 59.
This is not real “random” but gives a big spread of the cron task 'PATH="$PATH:/usr/local/bin/" pihole updateGravity’
until everybody enable the auto update cron task!! :wink:


#5

Could do a sleep on cron or the script (5min=300s):

sleep $((RANDOM % 300))


#6

We have an open pull request for this feature


#7

Between 2 and 4 AM in the morning.
Sounds reasonable.
Next feature request will be to set times using the UI :wink:

EDIT: I just now run the sed and now have:

55 2 * * 7 root PATH="$PATH:/usr/local/bin/" pihole updateGravity


#8

I agree with deHakkelaar: between 2 and 4 AM in the morning


#9

In Europe the actual time is changed to winter time.

In case of that I think the random automatic gravity update between 2 and 4 hr. is not so good idea.
There is a possibility that, when the random cron time is set between 2 and 3 hr, the update will be done 2 times.
But in March, when changing to summer time the gravity update is never done.

Maybe a time between 3:01 and 5 hr is a better choice
What do you think?


#10

Having just dealt with a load of stuff not properly set up for the rollback to UTC, I second this motion!


#11

I concur. I modified the pull request accordingly.


#12

This has been implemented and will be available in Pi-hole core v3.2.


#13