Add "PROXY protocol" Support -> quick win: DoH, DOT, DNSCrypt, Loadbalancing, dns rulesets with dnsdist

UPDATE:
“DNS X-Proxied-For” (XPF) is probably outdated, since today (20.03.2020) dnsdist also supports the PROXY protocol.

####################################################################

I have been running my own DNS infrastructure for some time now, including my own recursive DNS servers.
Currently the setup consists of dnsdist (DNS proxy) and unbound.
I have learned to appreciate dnsdist, because it allows to create DoH, DoT, DNSCrypt, load balancing and DNS rulesets very easily.

However, if you use dnsdist in front of pi-hole, only the dnsdist ip address will appear in the pi-hole logs. Since this problem is known (e.g. http Proxy), there is a draft “DNS X-Proxied-For” available. Dnsdist supports this in the newServer command (addXPF), but pi-hole does not understand the “DNS X-Proxied-For” entry.

It would be cool if we could include “DNS X-Proxied-For” in pi-hole, then you could take advantage of dnsdist.

https://tools.ietf.org/id/draft-bellis-dnsop-xpf-02.html

https://dnsdist.org/index.html
https://dnsdist.org/reference/config.html?highlight=newserver#newServer
https://dnsdist.org/guides/dns-over-https.html
https://dnsdist.org/guides/dns-over-tls.html

https://blog.powerdns.com/2018/03/30/dnsdist-1-3-0-released/

https://dnsprivacy.org/wiki/m/mobile.action#page/1278004

UPDATE:

Not sure whether it would be prudent to support this.

“DNS X-Proxied-For” is not a standard, not even an RFC yet. Its corresponding internet draft has been around since January 2017 and has expired in September 2018:

Since the “DNS X-Proxied-For” approach will probably not be further developed, the proxy protocol (v2) should be implemented in pihole.

dnsdist:


Nginx should also support this (proxy_protocol on;)

HAProxy:

A problem in this is that it handled by pihole-FTL/dnsmasq and I think dnsmasq is responding to the resolve as latest.

If we look at dnsdist which has several ways of providing the client IP address/subnet then Pihole has the extract the x-proxied-for address and use that, for logging and bookkeeping.
No need for replace, tagging, routing needed. Dnsdist gets the reply back and sends it to the client.

https://dnsdist.org/ is very nice software to front end a DNS server. Not just for this particular request but for all kinds of needs.

1 Like

Unfortunately, dnsdist (or other proxies) is currently not very easy to use with pihole, because the pihole gui only shows the source IP of dnsdist.
Of course you can create a workaround with policy based routing (I don’t think it works with dnsdist) but I would like to have an easier solution (I run everything with docker).

I thought XPF was the solution at first, but the proxy protocol is probably more common.

Unfortunately, a few things are still missing in order to make this work in the future.
My DoH setup:
Internet —(HTTPS,443)—> Haproxy —(HTTP,80, docker internal)—> dnsdist —(dns,53, docker internal)—> pihole --(dns,53, docker internal)—> unbound

My DoT setup:
Internet —(DoT,853)—> dnsdist —(dns,53, docker internal)—> pihole --(dns,53, docker internal)—> unbound

Missing:
Haproxy —(HTTP,80, docker internal)—> dnsdist. (only DoH)
- X-Forwarded-For support, only relevant if several services are to be used via port 443.
- dnsdist: https://github.com/PowerDNS/pdns/issues/8661

dnsdist —(dns,53, docker internal)—> pihole. (DoH and DoT)
- proxy protocol support
- dnsdist: https://github.com/PowerDNS/pdns/pull/8874
- pihole: nothing found for the proxy protocol yet, therefore this post

Which service would have to be adapted for the proxy protocol for pihole, pihole-FTL or dnsmasq? Which service provides the DNS port 53 for the clients?

Now dnsdist 1.5.0 (second release candidate) is available with Proxy Protocol and X-Forwarded-For support. Now pihole should follow.

I have tried to get go-mmproxy running in the pihole docker container, but unfortunately I haven't been able to get it to work. Installation was no problem, but the start with the init system s6-overlay is a problem.

1 Like

I would like to use pihole over the internet from the following devices:

  • iOS, MacOS, Androide, Firtzbox (a very well-known market for DSL-Rotuer in Germany)

Pihole is supposed to run on two virtual servers in the cloud as a docker container.

The use of port udp/53 is out of question from the following reasons:

  • udp/53 is not encrypted
  • on udp/53 already runs an authoritative dns server (powerdns), actually it runs behind dnsdist.
  • DNS Amplification Attack
    • In Germany you get a friendly email from the "Bundesamt für Sicherheit" (BSI) asking you to disable the open resolver on port 53.

Therefore only DoH or DoT can be considered. A VPN is not the perfect solution (energy consumption, no redundancy of the dns server), even the Fritzbox can not handle it.

DoH or DoT is (or soon) supported by the devices.
iOS/MacOS: https://www.techradar.com/news/apple-devices-will-get-encrypted-dns-in-ios-14-and-macos-11
Android: https://blog.cloudflare.com/enable-private-dns-with-1-1-1-1-on-android-9-pie/
Fritzbox : https://www.heise.de/newsticker/meldung/FritzOS-7-19-Bessere-WLAN-Verschluesselung-WPA3-fuer-AVMs-Fritzboxen-4597676.html

But I do not find any possibility to activate DoT or DoH in Pihole.

It is not a problem to activate a DoT/DoH proxy in front of pihole, but the client IP address is lost, you only see the IP address of the proxy in pihole gui.
There are instructions to solve this with routing/route maps, but this is only a very unpleasant workaround. Also in Docker this is difficult to implement (or do you have a good manual for it).

I also don't think the implication of DoT or DoH directly in Pihole is a good solution.
Also note that port 443, 53, 853 is not exclusively used by pihole but by other DNS services, so a DNS load balancer must be installed.
it would make much more sense if the developers had a look at the "Proxy ptrotocol". Then you could simply add haproxy or dnsdist (version 1.5 or higher) in front of it and you would have solved the problem.


https://dnsdist.org/advanced/proxyprotocol.html
https://dnsdist.org/guides/dns-over-https.html
https://dnsdist.org/guides/dns-over-tls.html

SOLUTION: Implementation of the "Proxy ptrotocol"

PS: With mmproxy I had unfortunately also no success: https://github.com/path-network/go-mmproxy

You can do that whit ZeroTier its end to end encryption.

and what does this have to do with pihole and dot/doh?

Neither DNS-over-TLS nor DNS-over-HTTPS do require authentication.

The solution you seem to be striving for will hence result in you running an open resolver, regardless of ports used.

The only reliable way, known to me, to safely access a cloud-based Pi-hole without publically exposing it as an open resolver is by means of a VPN.

As for your other request:

You could explore EDNS(0) to fix this, see Support for add-subnet option from dnsmasq (ECS/EDNS0 Client Subnet).

An open resolver is not a problem with DoT/DoH, it is only a problem with udp/53 because a "DNS Amplification Attack" can be performed, as soon as TCP is used the problem does not exist.
For this reason only a authoritative-only name server should run on udp/53.

If you use DoH you can even use a custom URL (I already do), so guessing the URL is difficult.
But even if somebody uses my open resolver over TCP, this is not a security problem.

EDNS I have a closer look at, dnsdist can do that too.
https://dnsdist.org/advanced/ecs.html

EDNS works :slight_smile:

1 Like

@DanSchaper @DL6ER
Here the desired feature requests for the proxy protocol.

Don't ping me. I ignore people that do it.

Can you provide some example for what is on the wire when using the proxy protocol? So far, there is nothing about how this is actually realized. The EDNS(0) data is shiped within the very same DNS request so working on this is fairly straightforward. Does "the proxy protocol" fit equally nice into the overall DNS framework?

I have had a quick look at the implementation PR of the procy protocol. It is a +2,527 lines of code change. Unfortunately, the readability of the code adding the proxy protocol to PowerDNS is ... well ... let's restirct ourselves to say that there is not a single line of comments in the source code unit that defines addProxyProtocol. This really makes it hard if not impossible to understand what they are doing there without further reading elsewhere.

Could you provide us with some examples like recorded pcaps so we can check if it is feasible to implement this in FTL? Another question is: What happens if you switch on this feature? Does Pi-hole still work as expected? If so, then they do not interfere much with the original DNS protocol and chances increase that we will be able to implement this in FTL.

I should, nevertheless, also mention that it is currently (and in the foreseeable future) beyond our capabilities to implement this when it turns out to be a lot of work. An external contribution to add this is obviously always welcomed. Note, however, that changes to the dnsmasq code needs to be submitted through the official dnsmasq project to ensure future compatibility so we can continue to easily integrate dnsmasq upstream updates.

We need Proxy Protocol v2 because it supports UDP.

https://developers.cloudflare.com/spectrum/getting-started/proxy-protocol/
https://www.haproxy.com/de/blog/haproxy/proxy-protocol/
https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt

I don't have a DNS server that supports the Proxy Protocol v2 with udp, I only use Postfix and Dovecot with the Proxy protocol with TCP.

Does Pi-hole still work as expected? -> No
Since the dns-port expects a proxy-protocol-header when the proxy protocol is enabled, normal clients can no longer connect. Only the proxies (e.g. dnsdist or haproxy) with activated proxy protocol can connect. But this is not bad, because clients in this setup never connect directly to pihole, but always via the proxy.

I have seen your links above. I have also read the technical description which is the third link. However, this does only describe the protocol in a very abstract way. There is no mentioning of how it would integrate with DNS. Hence my question.

If we don't have any examples, it will be close to impossible to realize this.

Let me rephrase my question, maybe I was unclear: Can you record a pcap in the configuration of your proxy in the way you would use it when Pi-hole could read the proxy protocol? I want to see what is sent to Pi-hole by the proxy. That's the only way I can think about how to even start looking at this.

I'm not familiar with any of the tools you are using. If I'd first have to setup a separated network segment with these services ... you see the overhead.

The proxy protocol has nothing to do with dns, it is a general approach to keep the client IP address when passing the protocol through a proxy.

Here is an example script for the encoding, I don't have a pcap.

https://github.com/ably/proxy-protocol-v2