For PiHole to work properly, it should be the only DNS in use, so yes. If the router is serving DHCP, then this DNS address will be passed on to all your network devices. Note that to get all your network devices using the new DNS after you set it in your router, you may have to clear DNS caches, restart, renew leases, etc (depends on the device).