The debug log on Docker installations is limited in scope. As you already found out, there is another process (e.g. another instance of dnsmaq or bind) running on your Synology NAS, but the debug log can't tell us which process it is. You need to check other apps/containers running on your NAS if they bind to port 53.
For anyone trying to find a workaround for this and trying to get port 53 free for you pihole
I found that 53 was being taken up by dnsmasq, which is what the synology uses for the DHCP server too (I had this enabled, because my router doesn't allow me to set custom DNS servers)
Having tried to find a way to configure that to not use 53 and failed, I decided to install the synology DNSServer package
this installs bind and disables the dnsmasq service from using port 53
now all I did was go to
/volume1/appstore/DNSServer/named/etc/named.conf
add the @ before appstore
I added some crap config to break the DNS server from starting, then rebooted
now port 53 is not in use on my synology and I can get pihole working