On a side note, I'd be reluctant to interfere with TTLs as configured by domain owners, i.e. I wouldn't recommend to overwrite DNS information by using cache-max-ttl and particularly cache-min-ttl if you aren't aware of the consequences. That said, your value of 5 minutes seems justifiable, not likely to trigger unwanted behaviour very often.
Pi-hole's cache optimiser and unbound's serve-expired, in the way you've configured it, very much do the same thing, so you could consider to disable serve-expired, or switch it to a slightly different behaviour, where it would only serve stale cached records if retrieving a fresh reply would take too long, e.g.:
server:
(…)
serve-expired: yes
serve-expired-ttl: 86400 # do not serve replies older than one day, in seconds
serve-expired-client-timeout: 1000 # serve expired replies when resolution takes longer than 1.0 seconds, in milliseconds
If a client requests resolution of an already expired cached record, Pi-hole would immediately serve stale data from its cache to clients while refreshing its cache by sending a DNS request to unbound, and unbound would start resolution and only serve stale data from its own cache if it wouldn't be able to complete resolution in one second.
This would increase likelihood that Pi-hole receives fresh, valid DNS records instead of just replacing its own stale record with a stale reply from unbound's cache.
unbound's prefetch option would trigger it to retrieve fresh replies if a record is served from cache requested within the last 10% of its TTL, i.e. before it is expired.
This feature won't have a chance to kick in if clients would cache unbound's reply themselves, as such a client would only send a DNS request once its TTL has expired, making it largely useless in a scenario where a local DNS server like Pi-hole aggregates traffic of an entire network.
Note that even with if clients would send DNS requests directly to unbound, prefetching would only kick in if one of the client would request a cached DNS record within the last 10% of its TTL.
It would depend on client behaviour if you'd actually benefit from this, and how much that would be able to improve Pi-hole and unbound already serving stale cached data.
You could either consider to disable unbound's prefetching, or to disable Pi-hole's cache altogether.
If you do the latter, you should be aware that Pi-hole won't be able to perform DNSSEC validation anymore, and all replies would be marginally slower, as Pi-hole would always have to forward allowed requests to unbound instead of serving them straight from its cache.