Use cached adlists on docker startup to improve startup times

I have read the following topic :

And the response is not really satisfactory, I have no notworking issue that I can solve, but just a slow internet connection (that is why I'm using pihole to avoid loading ads on a slow connection and to cache DNS a bit longer)

So every time I (re)start pihole, it is really not usefull to wait for all lists to be re-downloaded and parsed before starting dnsmasq/FTL... I did not find any option to avoid this.

Any help would be greatly appreciated !

Thanks in advance

Starting Pi-hole does not cause gravity to be updated, unless you are running Pi-hole in Docker. What commands are you using and what is the output?

I am using docker... That's why it's reloading all lists on container reboot. But I do have volumes persisting the data, using it for a while on reboot would be a nice feature.

and not to forget your question, I'm restarting using the docker command :
docker restart pihole (that's the name of my container)

Why do you require a restart of your container?

And what part of your request is already partially implemented?

Well, sorry for the tag "Partially implemented", I don't know how it ended up there, I'm quite new to discourse as you can see. None of it is partially implemented I think, except if you consider it's the default behaviour as jfb says when not running inside docker.
The piHole is part of my home lab and I'm restarting both it or the host itself on a regular basis for various reasons. As you can see others have asked for this already, I don't think listing the reasons I restart the container is pertinent here.

I appears your request is a duplicate of the existing feature request? If not, how is your request different?

What do you mean username checks out ? I'm not Job_e or whoever created the feature request I mentioned.

The guy in question has allegedly some issues making the download slow that someone told him to fix and that was enough to not consider the question at the time it seems. I have no such solvable issue : the download can only be slow, that is a fact of my connection.

Regardless if duplicate or not, and leaving aside your one-liner posts that don't adress the issue, can we go back to the question at hand please?

How is it you consider it's a good idea to wait potentially for a long time to start the DNS service while you already have cached data that you insist on refreshing before starting DNS service ?
You will happily run DNS service with 23h old cache, but you draw a line at serving cache that you don't know the age of ? I think it makes sense to sta6szrving dns queries using the existing cache, but I'm willing to hear arguments why I might be wrong.

I can understand that might be a low priority issue/feature, I can understand that if it weren't my first time on discourse I might have posted someplace else, but when developers and moderators only focus on not answering the issue I'm quite baffled here...

Scratch this, the previous post is closed, no way for me to reopen it or comment on it or anything. Only way I had to open a discussion on why pinhole is not using it's cache on startup when in docker was to create this post.

Where is there a 23 hour old cache? Items stay in the Pi-hole/dnsmasq cache for the duration of the TTL only.

Edit - what cache do you believe should be used on a restart of the docker container?

That's your problem then.

These appear to be written by the same person.
Are you running two accounts ?

Oh, it very much is.

Sharing your motivation will allow:
a) better judgement whether your Feature Request will benefit all or many users, or just a margin group with very specific and unique requirements
b) better understanding of the problem and thus a more targeted implementation
c) alternate suggestions to tackle your underlying problem

In many cases, c) would allow a user to find an immediate solution or workaround, avoiding having to wait for the next release, while sparing our developers from allocating their limited time to consider this issue at the same time. We all have day-time jobs and have to consider where to invest our spare-time.

This is why we try to understand your issue first.
So yes, your motivation matters.

It's in the title, and in the other thread : I would like for pihole to use the cached blocklists on startup instead of re-downloading all of them. Not talking about standard DNS cache for which TTL applies here.

Probably one of them was an email reply, didn't notice it shows up differently.
Both of them are me, only one account.

I apologize for my out of line responses and comments. I took your words to be an attack on Pi-hole and I get very defensive over it. That doesn't excuse my actions, it may be a native language issue.

As for caching, where does the 23 hour value come in to play?

Repeated restarting tells me there are other issues that need to be addressed but I can see the value in not running gravity and pulling down lists if they exist. How recent should the downloaded cached copies of the blocklists be before we download again? I don't want to have stale lists around and skip gravity purely because the files exist.

2 Likes

Apology accepted, I might also have been a bit rough, so I also apologize.

It's not an attack on piHole, I like it very much: I went out of my way to use it instead of the one integrated in my opnSense router....

The 23 hours delay is maybe from my imagination only, I was thinking gravity uptates it's list once per day with a cron or something like that.

On startup, my idea is just to delay the refreshing of the cached lists until all services are up and running, as long as it's done in the background I think it'll be fine.
If you get into calculating the age of the cached lists, I then see no reason to have another policy than the one you already use to refresh gravity when piHole is running normally. Meaning : if gravity usually checks lists hourly, then on startup, delay the refreshing until dns is running, and then if the cache is older than an hour, do the refresh as the last startup operation.

Hope I'm making sense, if not feel free to question me again, english isn't my mother tongue either :wink:

You're right, lots of issues I'm working on, with the docker host, the proxmox host, my network routing, and many more but not any other issue on piHole :wink: it's running fine, just making the process of fixing the rest a bit tedious with this slow restart, but still valuable to me so I keep it around :smiley:

That's a great idea. I think we may be able to expand on that for all installs and not just docker installs.

@PromoFaux and @DL6ER What about bringing up pihole-FTL first, get a resolver working and answering replies and then run gravity, that should swap in the new database with the blocks once it's complete. That would mean no blocking until the gravity run is complete but at least there's some kind of resolution working. If it takes 5 minutes on a xDSL line to download the domain lists, that's (currently) 5 minutes of no real DNS server working.

What's blocking you with Proxmox? DM me here if you have some questions or need help, we run Proxmox on a couple of Hetzner auction dedicated servers for Pi-hole.