I decided to change my config and use a static ULA address for my Pi-Hole using this guide: Use IPv6 ULA addresses for Pi-hole - #5 by DL6ER
Now my router uses RA to distribute the ULA address to clients. This works perfectly for iOS and MacOS clients, but Windows only seems to use the IPv6 DNS server when IPv4 is turned off on the client.
When dualstack is used, Windows only uses the IPv4 DNS server instead of both. This seems to be a problem with Windows 10/11 and the RFC 6106 implementation.
Interestingly enough, the Windows clients did seem to get both IPv4 and IPv6 DNS servers when the option "Enable IPv6 support (SLAAC + RA)" was turned on in Pi-hole, but that option was quite unstable on my Apple clients (the IPv6 server dissapeared often).