You are right, you can always restart the conatiner, but deleting these folders will likley render his current instance useless, as you strip Pi-hole from its configuration.
Trying to continue using an instance that has been crippled in that way will make you see errors that normally wouldn't occur.
Apologies if you didn't intend to continue using it after deletion (but why suggest deletion at all then?).
@sebtus:
Anyway, what's the content of /etc/dnsmasq.d/ and /etc/pihole/ in your container ?
Anything resembling the files as described in What files does Pi-hole use?