FTL webserver TLS handshake fails silently when forcing HTTPS-only (443s)

Help
1

Pi-hole FTL version: v6.7 (commit fa65a88f)
Web version: v6.6
Core version: v6.4.3
OS: Debian GNU/Linux (trixie), kernel 6.18.39+rpt-rpi-2712, aarch64 (Raspberry Pi 5)
Install method: standard package install

Summary

Setting webserver.port to force HTTPS-only (e.g. "443s,[::]:443s" or "80r,443s,[::]:80r,[::]:443s") causes the embedded webserver to accept TCP connections and log successful TLS certificate loading, but the TLS handshake itself fails on every connection attempt — both locally (127.0.0.1) and from remote clients. The only reliable fix is a full system reboot; restarting the pihole-FTL service (even repeatedly, even after reverting the config) does not resolve it once triggered.

Steps to reproduce

  1. Start from a working config with the default webserver.port = "80o,443os,[::]:80o,[::]:443os" (optional TLS) — confirmed working, HTTPS and HTTP both respond normally.
  2. Change webserver.port to "80r,443s,[::]:80r,[::]:443s" (or "443s,[::]:443s") directly in /etc/pihole/pihole.toml.
  3. sudo systemctl restart pihole-FTL.
  4. Attempt curl -v -k ``https://127.0.0.1/admin/.

Expected behavior

HTTPS connections succeed (as they do when the same certificate is used with 443os), and HTTP either redirects (80r) or is refused, depending on config.

Actual behavior

  • curl fails during the TLS handshake immediately after sending the Client Hello:
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* Send failure: Broken pipe
* TLSv1.3 (OUT), TLS alert, decode error (562):
* TLS connect error: error:0A000126:SSL routines::unexpected eof while reading
curl: (35) Send failure: Broken pipe
  • openssl s_client -connect 127.0.0.1:443 shows the TCP connection succeeds, but 0 bytes are read from the server ("SSL handshake has read 0 bytes and written 1551 bytes") — the server never responds to the Client Hello at all.
  • Plain HTTP (curl ``http://127.0.0.1/admin/) also starts failing with Recv failure: Connection reset by peer once this state is triggered — even after reverting webserver.port back to the original 80o,443os,... value and restarting the service again.
  • /var/log/pihole/FTL.log shows no error at all — it reports success on every restart:
INFO: Reading certificate from /etc/pihole/tls.pem ...
INFO: Using SSL/TLS certificate file /etc/pihole/tls.pem
INFO: - 0.0.0.0:80 (HTTP, IPv4, redirecting, OK)
INFO: - 0.0.0.0:443 (HTTPS, IPv4, OK)
  • ss -tlnp confirms the sockets are correctly bound and listening on both 80 and 443.
  • The certificate itself was verified valid and well-formed (openssl x509 -noout -dates -subject shows correct validity window; file contains 1 PRIVATE KEY block and 2 CERTIFICATE blocks as expected).
  • pstree/ps -eLf confirm the webserver thread is present and alive within the main pihole-FTL process (no crashed/missing child process).
  • No firewall rules (nft list ruleset, iptables -L -n -v) reference ports 80/443 at all — ruled out as a cause.
  • A local, unrelated Python http.server on a different port works fine at the same time, ruling out a system-wide networking/conntrack issue.
  • sudo systemctl stop pihole-FTL followed by a clean start (confirmed via ps aux that no process/thread survived the stop) does not clear the issue.
  • Only sudo reboot restores normal function — and even then, retriggering the 443s-only config change reproduces the exact same failure again (tested twice, reproducible).

Additional notes

  • CrowdSec + crowdsec-firewall-bouncer are running on this host, but cscli decisions list showed no active decisions during the failure, ruling that out as well.
  • This appears to point to a state/resource issue specific to the civetweb webserver thread that a graceful config reload/restart does not clean up, but a full process teardown (via reboot) does. Possibly related to how the optional-vs-mandatory TLS listener mode is initialized/torn down.

Happy to provide full journalctl -u pihole-FTL output, pihole.toml diffs, or run additional diagnostics if useful.

2

yeah that reboot-only recovery after a clean ftl stop is nasty. ive seen the mandatory tls path (443s) get weird while optional (443os) kept working, so staying on optional for now is probably the sane workaround if you need the ui. if you havent already, that repro belongs on a github.com/pi-hole/FTL issue. one more cheap try: regenerate the pem (or point webserver.tls at a fresh self-signed for a minute) and see if the silent handshake follows the cert file. crowdsec bouncer with empty decisions can still be odd sometimes, so a quick test with the bouncer stopped might be worth it too.

4

I reproduced your steps on my sandbox VM and did not experience any issues.

What package are we talking about?
Bc the official Pi-hole doesnt come in a software package.

Might want to post a debug token, like was requested in the help template, for the devs/mods to have a look?

5

Good catch, apologies for the imprecise wording. It's installed via the official install script (curl -sSL ``https://install.pi-hole.net`` | bash), not an OS package. Confirmed on the box: /etc/.pihole is a git checkout (contains .git, gravity.sh, automated install/, etc.) and the pihole binary lives in /usr/local/bin, matching the standard installer layout — no dpkg/apt package involved for Pi-hole itself (FTL and the web interface are the separate components that are installed as their own binaries by that same installer, if that's the "package" that caused confusion).

The debug token from the original post (https://tricorder.pi-hole.net/1gxJA0be/) should still be valid for another ~24h at the time of this reply — let me know if it's expired and I'll generate a fresh one.

Also following up on nite_route's suggestions above: going to try stopping the crowdsec-firewall-bouncer and regenerating the TLS cert before re-testing 443s, will report back with results.

6

Thanks for the suggestions — tried both, same failure in both cases.

CrowdSec bouncer stopped, then 443s enabled:

sudo systemctl stop crowdsec-firewall-bouncer

Confirmed stopped, then switched webserver.port to "80r,443s,[::]:80r,[::]:443s" and restarted FTL. Exact same result as before:

* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* Send failure: Broken pipe
* TLS connect error: error:0A000126:SSL routines::unexpected eof while reading
curl: (35) Send failure: Broken pipe

So the crowdsec bouncer isn't a factor here — same failure with it fully stopped.

Fresh self-signed cert:
With 443s still active from the test above, removed the existing tls.pem and let FTL regenerate one:

sudo rm /etc/pihole/tls.pem
sudo systemctl restart pihole-FTL

Confirmed in FTL.log that a genuinely new cert was created:

INFO: Created SSL/TLS certificate for pi.hole at /etc/pihole/tls.pem
INFO: Reading certificate from /etc/pihole/tls.pem ...
INFO: Using SSL/TLS certificate file /etc/pihole/tls.pem

Same TLS handshake failure afterwards, identical error output as above. So a fresh cert doesn't change anything either.

At this point I've ruled out: the cert itself (valid both times, checked structure/dates on the original one too), CrowdSec (stopped bouncer, no active decisions either), firewall rules (nft/iptables both empty for 80/443), and now cert regeneration. The webserver thread is confirmed alive (checked via ps -eLf, it's a thread inside the main FTL process, not a separate crashed process) and the socket is listening (ss -tlnp shows it bound), FTL.log reports "OK" on both ports every time — it just never completes the TLS handshake once 443s (mandatory) replaces 443os (optional). Reverting to 443os after each test restores things back to a normal state (well, restores it after a full reboot — restarting the service alone doesn't do it once triggered, that's the nastier part I mentioned earlier).

Still on 443os for daily use, will hold off on further live testing on this box for now unless there's something specific worth trying — happy to if it'd help narrow it down further.

7

I see a chance here that this could be already fixed in development.

The current stable release (v6.7) uses mbedTLS as the TLS library. Its integration with civetweb is not that well tested and has casued us multiple issues in the past.
The next release (v7) will use openSSL as the TLS library with a custom-built TLS terminator that with also support both HTTP/2 and HTTP/3 (QUIC), while civetweb itself is built with NO_SSL and our terminator forwards all requests.

You may test this by switching to the development branch via pihole checkout dev.
development currently carries two breaking changes - dns.blockESNI and the flush/arp API route are gone, and the webserver.log format changed plus logrotate now requires SIGUSR2 - but nothing touches the databases, so pihole checkout master takes you straight back if you need it.

8

Tried the dev branch, thanks for the pointer — interesting result, the TLS layer itself does seem fixed, but there's a new, more specific failure behind it.

Setup: sudo pihole checkout dev → confirmed FTL branch: development, FTL version: vDev-c280cc6. Sanity check on the untouched config (443os) worked fine over plain HTTP first. Then switched webserver.port to "80r,443s,[::]:80r,[::]:443s" and restarted.

Good news — TLS handshake now completes:

* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Request completely sent off
< HTTP/2 502

No more "Broken pipe" / "unexpected eof" during the handshake like on stable v6.7. So the new OpenSSL terminator does seem to fix the original mbedTLS/civetweb handshake issue.

But now getting a 502 from the terminator. FTL.log confirms the new architecture is active:

INFO:   - 0.0.0.0:443 (HTTPS, terminator)
INFO: TLS terminator listening on *#443, forwarding to 127.0.0.1:33101

So I went straight at the backend, bypassing the terminator entirely:

curl -v http://127.0.0.1:33101/admin/

Result: Recv failure: Connection reset by peer — same with / instead of /admin/. ss -tlnp confirms civetweb is listening on 33101 (fd=41 on the FTL pid), so it's not that the backend never started — it accepts the TCP connection, then resets once the HTTP request comes in. And just like the original stable-branch bug, FTL.log shows nothing at all about these failed requests — not even at the interesting timestamps, silent every time.

Also tried forcing HTTP/1.1 through the terminator instead of letting curl offer h2:

curl -k --http1.1 https://127.0.0.1/admin/

→ Empty reply from server this time, instead of the 502. So the terminator's own error behavior isn't even consistent depending on protocol negotiation, but the common thread across all of it is: civetweb itself resets the connection on every plain-HTTP request to its own backend port, with nothing logged. That part looks unchanged from stable, just now one layer removed from TLS.

So my read: the terminator/OpenSSL work does resolve the original TLS handshake symptom, but there's a separate, still-open issue in civetweb's request handling when it's running NO_SSL on a backend port — unrelated to which TLS library sits in front of it. Happy to test anything more specific against 33101 directly if that'd help narrow down the civetweb side of it.

Will revert to stable/master for daily use for now (pihole checkout master + reboot, same recovery as before since the FTL state seems to need a full boot to clear once this triggers).

9

For what's it worth :

I have been using HTTPS only since the release of Pi-Hole v6 and never had issues with it while running on a Raspberry Pi 3B and Debian 12 with browsers like Pale Moon/Firefox/LibreWolf :slight_smile:

10

Any progress?

I reproduced and found no issues?

$ pihole version
Core version is v6.4.3 (Latest: v6.4.3)
Web version is v6.6 (Latest: v6.6)
FTL version is v6.7 (Latest: v6.7)

$ sudo pihole-FTL --config webserver.port '80r,443s,[::]:80r,[::]:443s'
80r,443s,[::]:80r,[::]:443s

$ sudo ss -nltup | grep 'Netid\|pihole'
Netid State  Recv-Q Send-Q Local Address:Port Peer Address:PortProcess
udp   UNCONN 0      0            0.0.0.0:53        0.0.0.0:*    users:(("pihole-FTL",pid=2246,fd=21))
udp   UNCONN 0      0            0.0.0.0:123       0.0.0.0:*    users:(("pihole-FTL",pid=2246,fd=40))
udp   UNCONN 0      0               [::]:53           [::]:*    users:(("pihole-FTL",pid=2246,fd=23))
udp   UNCONN 0      0               [::]:123          [::]:*    users:(("pihole-FTL",pid=2246,fd=41))
tcp   LISTEN 0      200          0.0.0.0:80        0.0.0.0:*    users:(("pihole-FTL",pid=2246,fd=36))
tcp   LISTEN 0      32           0.0.0.0:53        0.0.0.0:*    users:(("pihole-FTL",pid=2246,fd=22))
tcp   LISTEN 0      200          0.0.0.0:443       0.0.0.0:*    users:(("pihole-FTL",pid=2246,fd=37))
tcp   LISTEN 0      200             [::]:80           [::]:*    users:(("pihole-FTL",pid=2246,fd=38))
tcp   LISTEN 0      32              [::]:53           [::]:*    users:(("pihole-FTL",pid=2246,fd=24))
tcp   LISTEN 0      200             [::]:443          [::]:*    users:(("pihole-FTL",pid=2246,fd=39))

$ openssl s_client -connect 127.0.0.1:443 -servername pi.hole </dev/null 2>/dev/null | openssl x509 -noout -text
Certificate:
[..]
        Subject: CN = pi.hole
[..]
            X509v3 Subject Alternative Name:
                DNS:pi.hole

$ curl -kIH "Host: pi.hole" http://127.0.0.1/
HTTP/1.1 308 Permanent Redirect
Location: https://pi.hole/

$ curl -kIH "Host: pi.hole" https://127.0.0.1/
HTTP/1.1 308 Permanent Redirect
Location: /admin/

$ curl -kIH "Host: pi.hole" https://127.0.0.1/admin/
HTTP/1.1 302 Found
Location: /admin/login

$ curl -kIH "Host: pi.hole" https://127.0.0.1/admin/login
HTTP/1.1 200 OK
11

Couldnt find the right syntax to follow links (-L) at first :wink:

$ curl -kIL --resolve pi.hole:80:127.0.0.1 --resolve pi.hole:443:127.0.0.1 http://pi.hole/
HTTP/1.1 308 Permanent Redirect
Location: https://pi.hole/
[..]
HTTP/1.1 308 Permanent Redirect
Location: /admin/
[..]
HTTP/1.1 302 Found
Location: /admin/login
[..]
HTTP/1.1 200 OK