Since this is installed on a Beaglebone, it recreates the SoftAP0 file on reboot no matter what you do, even if you comment out the offending sections.
The only way I could get it to stop modifying the file (the duplicate dnsmasq.leases file was causing issues previously) is to use chattr +i to make the file immutable.
In this case, I truncated the file and made it immutable so that the file exists but can't be overwritten by the system. Pihole seems to be working okay now.