So you've configured Pi-hole to use your router as its upstream:
[dns]
upstreams = [
"192.168.0.1#853"
] ### CHANGED, default = []
As I understand it, unbound is listening on port 853 on your router.
Pi-hole only talks plain DNS/Do53, i.e. it does not encrypt its DNS requests. Configuring it to use an upstream port 853 does not change this.
As port 853 is indeed reserved for DoT usage, you should consider to change your upstream to 192.168.0.1#53.
As for your observation:
You wouldn't, as Pi-hole v5 didn't log that message, so that doesn't mean that the reported TCP connection failure hasn't happened already with v5.
As for the potential causes, we've intensively analysed and identified that as being caused by specific behaviour of unbound, see Two Pi-hole instances, One with "failed to send..." - #44 by DL6ER.
There is currently nothing that Pi-hole could do to improve this.
The condition has been reported to unbound, where some potential measures are currently explored. Increasing unbound's incoming-num-tcp (e.g. to 20) is reported to mitigate the issue.
You may consider to contribute your observation over at failed to send TCP(read_write) packet (Connection prematurely closed by remote server) · Issue #1237 · NLnetLabs/unbound · GitHub.