When starting the pihole container docker will open the firewall to allow external connections to port 53, assuming you're doing something like -p 53:53 or
ports:
- 53:53
to forward host connections on 53 to the pihole's 53.
I'm not a docker expert, but as far as I know there is no simple solution. You can try changing around the order you start docker and ufw, but I don't know if that will help. You can tell docker not to mess with iptables, but that will cause lots of other problems.
I manage this by running dnsdist at the host level, and let it listen on port 53. Then dnsdist passes queries to the pihole container, and docker doesn't mess with the firewall on port 53, because docker is run with
ports:
- 127.0.0.1:8053:53
Another option is to configure your clients to connect to your VPS with DoH or DoT, and close port 53. For that to work, you'll have to setup something to listen for DoH or DoT (such as dnsdist).
And as said, if you only want to use this from home, then you could setup a VPN from your router to your VPS, and limit port 53 access to just the VPN. That can be tricky, too, because you don't want all of your traffic going over the VPN, just traffic destined for the VPS.