No need to reset your modem.
If you are going to manually set an IPv6 address as DNS server on your Win10 machine, start by listing current IPv6 addresses on your RPi 4B by running the following command:
ip -6 address show eth0
Replace eth0 with the correct interface as required.
Then pick an IPv6 address from that list, but avoid public IPv6 addresses (2000::/3 range), and use that as your Win10 DNS server's IPv6 address.
The same address should be configured as Pi-hole's IPv6 address.
If it isn't already, run
pihole -r
and choose Reconfigure.