So I did some hardcore digging.
The issue seems to have been this in the error.log:
gw_backend.c.475) unlink /run/lighttpd/php.socket-0 after connect failed: Connection refused
Very hard to find but I came across this.
Apparently you are not allow to nest "$SERVER["socket"]", and so I removed this from my external.conf:
$HTTP["host"] == "my.local.ip" { }
I had copy-paste the external.conf from somewhere, I did not figure that out myself.
After removing that line (and thus unnesting the $SERVER socket) and restarting lighttpd, it all seems to work again. Tough find, hopefully useful for someone who has the same issue in the future.