Yeah you beat me to it.
Was bout to post below through trial error 
pi@noads:~ $ sudo netstat -nltup | grep 'Proto\|lighttpd'
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 5977/lighttpd
tcp6 0 0 :::80 :::* LISTEN 5977/lighttpd
pi@noads:~ $ sudo mkdir /etc/lighttpd/certs
pi@noads:~ $
pi@noads:~ $ cat pi.hole.crt pi.hole.key | sudo tee /etc/lighttpd/certs/pi.hole.pem
[..]
pi@noads:~ $ sudo chmod 600 /etc/lighttpd/certs/pi.hole.pem
pi@noads:~ $
pi@noads:~ $ sudo nano /etc/lighttpd/external.conf
$HTTP["host"] == "noads.dehakkelaar.nl" {
# Ensure the Pi-hole Block Page knows that this is not a blocked domain
setenv.add-environment = ("fqdn" => "true")
# Enable the SSL engine with a LE cert, only for this specific host
$SERVER["socket"] == ":443" {
ssl.engine = "enable"
# ssl.pemfile = "/etc/letsencrypt/live/pihole.example.com/combined.pem"
# ssl.ca-file = "/etc/letsencrypt/live/pihole.example.com/fullchain.pem"
ssl.pemfile = "/etc/lighttpd/certs/pi.hole.pem"
ssl.honor-cipher-order = "enable"
ssl.cipher-list = "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"
ssl.use-sslv2 = "disable"
ssl.use-sslv3 = "disable"
}
$SERVER["socket"] == "[::]:443" {
ssl.engine = "enable"
# ssl.pemfile = "/etc/letsencrypt/live/pihole.example.com/combined.pem"
# ssl.ca-file = "/etc/letsencrypt/live/pihole.example.com/fullchain.pem"
ssl.pemfile = "/etc/lighttpd/certs/pi.hole.pem"
ssl.honor-cipher-order = "enable"
ssl.cipher-list = "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"
ssl.use-sslv2 = "disable"
ssl.use-sslv3 = "disable"
}
# Redirect HTTP to HTTPS
$HTTP["scheme"] == "http" {
$HTTP["host"] =~ ".*" {
url.redirect = (".*" => "https://%0$0")
}
}
}
pi@noads:~ $ sudo service lighttpd restart
pi@noads:~ $
pi@noads:~ $ sudo netstat -nltup | grep 'Proto\|lighttpd'
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN
tcp6 0 0 :::80 :::* LISTEN
tcp6 0 0 :::443 :::* LISTEN
pi@noads:~ $ curl -Ivk https://[::1]
* Rebuilt URL to: https://[::1]/
* Trying ::1...
* TCP_NODELAY set
* Connected to ::1 (::1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd; CN=noads.dehakkelaar.nl
* start date: Sep 8 20:18:09 2019 GMT
* expire date: Sep 5 20:18:09 2029 GMT
* issuer: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd; CN=noads.dehakkelaar.nl
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> HEAD / HTTP/1.1
> Host: [::1]
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Content-type: text/html; charset=UTF-8
Content-type: text/html; charset=UTF-8
< Date: Mon, 11 May 2020 03:13:38 GMT
Date: Mon, 11 May 2020 03:13:38 GMT
< Server: lighttpd/1.4.45
Server: lighttpd/1.4.45
<
* Curl_http_done: called premature == 0
* Connection #0 to host ::1 left intact
EDIT: added -k argument for curl for self signed certs.