Prefetch Popular Domains to Improve Cached Response Frequency

[Matt]

@Coro

• Does unbound do prefetching? This is what @yubiuser suggests

Yes.

@Coro

• What would be the benefit of adding (another layer of) prefetching to Pi-hole?
  (I write "another layer" but this is obviously only meant for the unbound users)

I'm working to gather data to indicate whether or not this is the case, but prefetching at the Pi-hole level could actually trigger additional prefetching (in a good way) at the Unbound level - based on the publicly shared Unbound prefetching algorithm. This assumes Pi-hole's algorithm functionally mimics Unbound's, and caching is done at the Pi-hole. If Pi-hole's algorithm differs from Unbound's, in a complementary fashion, it's possible that it could further increase the overall cached response rate.

Additionally, you highlighted the other obvious benefit - non-Unbound users would be able to capitalize on a DNS prefetching on their Pi-hole.

@Coro

I see only two effects of implementing prefetching in Pi-hole:

1. PRO: The delay for the queries (every hour?) is decreased by some 10-100 milliseconds.
2. CON: Pi-hole has to implement their own algorithm to decide what is a "hold" domain and what is a "cold" domain. May be how often the domain is queried in total (> 2 per minute) or relative to others (> 2% of total number of queries) or or or...

1. Does your network only see one query per hour? If not, better cache return ratios are likely to be more frequent than hourly... See Bucking_Horn's math below your comment for an estimate on the impact to the average user. I'm still gathering my own statistics, but using an existing post from sawsanders as a reference, he had 13137 recursive look-ups in a 6-day period, or ~90 recursive look-ups per hour (assuming they were evenly distributed over a 24 hour period, and not preferentially happening during hours where people are actually using the network). For me, at least, an unbound recursive look-up takes somewhere between 100 and 1000 ms. Anything done to reduce the number of recursive lookup responses in a day is beneficial. A 1s lookup has a noticeable impact on page loading.

2. Yes, the "con" to every feature request is that it will require development. That's the basic premise of a feature request. The beautiful thing is that once the basic prefetching code is in place, algorithms can be tuned to optimize prefetching over time. The system can start with a basic pre-fetching system like either or both of the algorithms I suggested in my first post, but over time can evolve. This is very much not the scope of the request today, but if a feature like this were to be implemented and allowed to evolve over time, prefetching would be a perfect candidate for a machine learning algorithm - they're designed for pattern recognition and prediction.