Support for add-subnet option from dnsmasq (ECS/EDNS0 Client Subnet)

Feature Requests Implemented
1

The ask: Add support within FTLDNS for parsing the IP address/subnet passed from another dnsmasq client via add-subnet=32,128 and potentially add-mac. Asking that FTLDNS support parsing of ECS (EDNS0 Client Subnet) data from an inbound query on the requesting side.

The reasoning:
I have an OpenWrt router running dnsmasq for my three client subnets. It forwards queries to my Pi-hole server, which is running on a separate VM. Traditionally, the recommendation by many in the Pi-hole community is to have dnsmasq on OpenWrt just hand out the Pi-hole IP via DHCP (option 6). Technically, this works and I realize it fits the majority of use-cases.

My use-case is perhaps a little more unique. On my OpenWrt router, dnsmasq is updating local ipsets (using --ipset) for QoS on my router. I cannot seem to come up with a workable option around this, so bypassing dnsmasq as my first-contact DNS on OpenWrt is not an option for me at this point.

I am trying to switch back to Pi-hole from NextDNS. My previous configuration was running the NextDNS client on my OpenWrt router, still having dnsmasq as the first-contact DNS, then forwarding to the NextDNS client on an alternate port number. As long as dnsmasq was configured with these options, the actual requesting client's IP (and MAC) were passed through dnsmasq to the NextDNS client:

add-mac
add-subnet=32,128

This allowed every query in NextDNS' console to show the client IP/hostname (via local resolution). Ref: DNSMasq Integration · nextdns/nextdns Wiki · GitHub I believe this is currently handled by parsing the ECS (EDNS0 Client Subnet) data from the inbound DNS query (coming in to FTLDNS from dnsmasq, in this case).

My hope is that FTLDNS could be updated to check for the same data in an inbound query and indicate the actual client IP that sent the request as opposed to the IP address of the OpenWrt server for every query. I think this could simplify configuration for a lot of Pi-hole users, too, even if the use-case isn't as technically required as in my case.

2

FTL supports everything dnsmasq offers as well. There are no limitations I'm aware of. You can modify configuration files in /etc/dnsmasq.d (best to create your own files) and restart FTL. It will pick up your options and you're good to go. Even better, Pi-hole always ships with the most recent dnsmasq version so you have features (and bug fixes!) typically a lot earlier than through regular OS releases.

3

I appreciate the response, but I think the context got lost. The request is to have FTLDNS parse and handle the ECS (EDNS0 Client Subnet) data that would be inbound from another dnsmasq client (downstream) that has those options set.

4

Ah, thank you for editing your post and apologies for not seeing this on the first read.

Let me summarize this with my own words to see if I got it right this time. I will enumerate it so you can quickly say which points are wrong (if any :slight_smile: )

  1. Your clients ask your router as DNS server (running dnsmasq)
  2. Your router forwards to the Pi-hole
  3. Hence, the Pi-hole only ever shows the router as client

Your request now is that, if there is a ECS record (typically there is none), Pi-hole should discard the "actual" source IP and use the ECS-provided one instead (because it will be more useful in this case)

1 Like
5

Exactly--you nailed it. Great summary! :smiley:

6

Great! So I have one remaining question and then we should leave the stage for the experts:

I do understand add-subnet=32,128, but why add-mac (I see you marked it as potential)?

7

Great question, and to be honest, I'm not sure if it would be needed. I see NextDNS client references it here: nextdns/query.go at 89739d863f346f8935c5b5b9b96c148a5010e948 · nextdns/nextdns · GitHub. I am guessing they use it as a fallback to do a "last-ditch" ARP for the IP in the case where the subnet did not come over in the ECS record.

** Update **
Thinking through this a bit more, it would stand to reason that add-mac would also come into play if the ECS is > /32 for IPv4 or > /128 for IPv6. If either of those cases is true, what was received in the ECS would be a true subnet as opposed to a single host IP (e.g. FTLDNS gets a 192.168.0.0/24 from ECS instead of 192.168.0.57/32). That would not be helpful for narrowing the query down to a single IP/hostname, so it seems ARPing for the provided MAC would be the next available option for obtaining a single IP.

8

but would certainly also introduce more work = delay

9

Agreed, but it might be a trade-off one is willing to make in order to get a more clear picture of the device making the query. I'm of the mindset all of this, even if implemented, would be more of an advanced/opt-in type of configuration. For those that understand it and opt for using it, they could make that judgement call for the added milliseconds of an additional ARP.

My expectation would be that FTLDNS would also strip ECS from the query before forwarding upstream for security reasons. But even that could be an opt-in/opt-out feature.

10

I must say I like this idea and see a lot of potential value in it (given routers can be made to send ECS data).

Over the past two hours, I went back and forth between RFC 6891 and RFC 7871 and finished a first implementation of the Extension Mechanisms for DNS ("EDNS(0)"). FTL should now be able to extract the proper address + prefix-length from the incoming queries.

I'm not yet sold on the MAC feature as there doesn't seem to be a lot of publicly available documentation. You can find details spread across mailing lists mentioning unpleasant things like

its value is implementation specific

@_FailSafe It would be great if you could do some initial testing, whether this works as expected in your environment.

Please run

pihole checkout ftl new/edns0

and continue using Pi-hole as usual. Check /var/log/pihole-FTL.log for lines starting in EDNS0.

Note: This is only the very first step. There is much work left to be done because FTL is already interpreting the DNS packet while it is received (for performance reasons).
This has the consequence that everything (like detecting if a given domain should be blocked for a given client, etc.) is already done and set before the additional records of the query (containing the EDNS0 information) are received/analyzed...

3 Likes
11

I pulled that branch on both of my Pi-holes. Here's a sample of what I'm seeing in the pihole-FTL logs now:

[2020-07-20 15:23:15.738 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:15.738 57621M] EDNS0: Identified option CLIENT SUBNET XXXX:YYYY:ZZZZ:2345:70a9:cedb:910a:c690/128
[2020-07-20 15:23:15.738 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:15.738 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:15.739 57621M] EDNS0: Identified option CLIENT SUBNET XXXX:YYYY:ZZZZ:2345:70a9:cedb:910a:c690/128
[2020-07-20 15:23:15.739 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:15.781 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:15.781 57621M] EDNS0: Identified option CLIENT SUBNET XXXX:YYYY:ZZZZ:2345:70a9:cedb:910a:c690/128
[2020-07-20 15:23:15.781 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:16.106 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:16.106 57621M] EDNS0: Identified option CLIENT SUBNET 192.168.99.120/32
[2020-07-20 15:23:16.106 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:16.357 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:16.357 57621M] EDNS0: Identified option CLIENT SUBNET XXXX:YYYY:ZZZZ:2345:1cce:b4b0:9702:281c/128
[2020-07-20 15:23:16.357 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:16.357 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:16.357 57621M] EDNS0: Identified option CLIENT SUBNET XXXX:YYYY:ZZZZ:2345:1cce:b4b0:9702:281c/128
[2020-07-20 15:23:16.357 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:16.358 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:16.358 57621M] EDNS0: Identified option CLIENT SUBNET XXXX:YYYY:ZZZZ:2345:1cce:b4b0:9702:281c/128
[2020-07-20 15:23:16.358 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:16.500 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:16.500 57621M] EDNS0: Identified option CLIENT SUBNET 192.168.99.120/32
[2020-07-20 15:23:16.500 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:16.501 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:16.501 57621M] EDNS0: Identified option CLIENT SUBNET 192.168.99.120/32
[2020-07-20 15:23:16.501 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:16.502 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:16.502 57621M] EDNS0: Identified option CLIENT SUBNET XXXX:YYYY:ZZZZ:2309:1110:c38d:5708:7fb6/128
[2020-07-20 15:23:16.502 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:16.502 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:16.502 57621M] EDNS0: Identified option CLIENT SUBNET XXXX:YYYY:ZZZZ:2309:1110:c38d:5708:7fb6/128
[2020-07-20 15:23:16.502 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:17.318 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:17.318 57621M] EDNS0: Identified option CLIENT SUBNET XXXX:YYYY:ZZZZ:2345:f41d:94bd:fd03:a09f/128
[2020-07-20 15:23:17.318 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:17.318 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:17.318 57621M] EDNS0: Identified option CLIENT SUBNET XXXX:YYYY:ZZZZ:2345:f41d:94bd:fd03:a09f/128
[2020-07-20 15:23:17.318 57621M] EDNS0: Originally recorded client was 192.168.50.5
[2020-07-20 15:23:17.318 57621M] EDNS0: Identified option MAC ADDRESS
[2020-07-20 15:23:17.318 57621M] EDNS0: Identified option CLIENT SUBNET XXXX:YYYY:ZZZZ:2345:f41d:94bd:fd03:a09f/128
[2020-07-20 15:23:17.318 57621M] EDNS0: Originally recorded client was 192.168.50.5

This is working exactly as I would expect, to be honest. Let me know what I can do to further help in bringing this feature into reality. I am very happy you feel it is worth the time and effort. :+1:

2 Likes
14

Remaining steps done as well. It was a bit painful to rearrange the DNS header analysis stuff, but I think I managed to do it without breaking anything. Please update again and check if it works now. The next steps are

  1. monitor for strange side-effects (I actually don't really expect any), and
  2. disable the debugging output you've quoted by default.

This definitively is going to be in the next version of Pi-hole (well, not the one about to be released, but the one after that) if nothing strange comes up in further tests.

1 Like
15

This is fantastic, really. I did update again and both Pi-holes are working beautifully now. I have my FTLDNS configuration set up to do reverse IPv4 and IPv6 lookups against dnsmasq on my OpenWrt box. Because I also run ip6neigh on my OpenWrt box, I have IPv6 SLAAC reverse lookups working there.

So with the updates you just made for EDNS0 processing, I am getting my ipsets populated on OpenWrt for QoS purposes, client IP/hostname reporting against queries in my Pi-hole VMs, and the piece of mind of having my Pi-holes querying externally via dnscrypt-proxy. I'm a happy camper with a big smile on my face right now. :slight_smile:

View of Pi-hole web console:

Screen Shot 2020-07-20 at 3.47.03 PM
Screen Shot 2020-07-20 at 3.47.03 PM1275×1502 222 KB

(Look at all that samsung tracking getting blocked... :clap:)

I will obviously be using the heck out of this, so I will be sure to report back on any hiccups. Although I have a good feeling this will be a continued success!

2 Likes
25

Another candidate for the most awesome new feature of the year. Pi-hole and especially their core developers definitely deserve credit and recognition for the awesome work they do almost every day!

COME ON. This (complex!) feature just been requested only yesterday and they implemented it on the same day. And the user requesting this already replied that this is doing exactly what they wanted it to do.

You cannot tell me that this is what you can expect from an average open source project.
Run by volunteers.
You are absolutely top notch guys!

Service, quality and support - everything just rocks in this project.

2 Likes
27

So I invested some more work into this to "finish" the feature. We can now also parse EDNS(0) cookies and a few other things even if we don't do anything with them (yet).

Concerning MAC addresses, this is really not standardized at all, I implemented it now in the exact same way dnsmasq has it. It is very possible that dnsmasq is the only DNS client that sends this packet. I implemented support for add-mac and add-mac=text. The former is more efficient, so to be preferred. I did not add support for add-mac=base64 as I couldn't be bother to code a base64 decoder. I don't think this is much of a loss.

It would be nice if you could update and do some more testing - are MAC addresses correctly interpreted and shown? Please add

DEBUG_EDNS0=true

to /etc/pihole/pihole-FTL.conf (create if it does not exist) so you get the debug log lines.

1 Like
28

Confirmed. I spot checked around two dozen MAC/IP combinations from the logs and the interpreted MAC addresses marry up to both IPv4 and IPv6 addresses perfectly. I validated the matches based on the arp table and ip6neigh whois command on my OpenWrt box.

I did not modify my OpenWrt dnsmasq configuration to add-mac=text, but could do so if you would like me to test that specifically.

4 Likes
29

No need to. Thanks for testing this so thoroughly!

1 Like
30

Absolutely! I appreciate the time and attention to this. Testing is the least I can do. If you have any other test-cases you would like me to run through, I'll be very happy to do that.

For what it's worth, it is still humming along very nicely. I thought maybe it hit an issue last evening because one of my Pi-holes got stuck in a loop trying to resolve an internal IPv6 PTR record. It ended up taking both of my Pi-holes to their knees. Was taking around 4 seconds per query to resolve any requests from my client machines.

Turned out to be a self-inflicted DNS forwarding loop with my OpenWrt dnsmasq. I forgot to modify my OWRT dnsmasq config to not forward internal IPv6 PTR lookups since it handles those locally via ip6neigh. Anyway, 15 mins of troubleshooting and >120,000 DNS queries later, I limped away from that one with a bruised ego. But, I am very happy it was not related to the changes you made for EDNS0 handling. :slight_smile:

** 48+ Hours Update **
Still running smoothly as can be on both of my Pi-holes. No issues to report.

6 Likes
32

Zero issues to report on this branch. Both Pi-holes have been running solid. I will not keep updating this thread with uptime reports unnecessarily, but still happy to do any additional testing needed. :+1:

@DL6ER Thanks again for the efforts put into this. I think it will be a great feature that will simplify a lot of configurations going forward!

3 Likes
33

@DL6ER Would it be feasible to have the client subnet included in the query[<record type>] <fqdn> from <requester IP> output in pihole.log (and subsequently via pihole -t)?