Same on the master branch:
I moved the category back to General.
So I've been playing around with this a little and the problem extends to even the known query types. All queries are affected if the answer doesn't fit one of the predefined reply types.
But even the DS and DNSKEY queries don't have DNSSEC as a reply type here for some reason.
There aren't any reply types that would fit any of these queries here, so N/A is better than nothing I guess, but something more descriptive like DATA would be nice just to have any feedback about the result of the query. Also all affected queries were forwarded again instead of being answered from cache. The screenshot doesn't show this but this is the second round of queries (testscript below). The response time is again always 0.0 ms even though the queries were forwarded.
# Test script to quickly test for arbitrary resource records
pihole="192.168.178.15"
for i in a aaaa srv soa ptr txt naptr mx ds rrsig dnskey ns svcb https
do
if [ "$i" = ptr ]; then
ip4="$(dig +short a netmeister.org)"
ip6="$(dig +short aaaa netmeister.org)"
echo "$(dig +noall +answer @${pihole} -x ${ip4})"
echo "$(dig +noall +answer @${pihole} -x ${ip6})"
fi
if [ "$i" = svcb ]; then
j="TYPE64"
elif [ "$i" = https ]; then
j="TYPE65"
else
j="$i"
fi
echo "$(dig +noall +answer @${pihole} ${j} ${i}.dns.netmeister.org)"
done
# Output of above script
a.dns.netmeister.org. 1158 IN A 166.84.7.99
aaaa.dns.netmeister.org. 1168 IN AAAA 2001:470:30:84:e276:63ff:fe72:3900
srv.dns.netmeister.org. 1246 IN SRV 0 1 80 panix.netmeister.org.
soa.dns.netmeister.org. 1254 IN SOA panix.netmeister.org. jschauma.netmeister.org. 2021072312 3600 300 3600000 3600
99.7.84.166.in-addr.arpa. 1255 IN PTR panix.netmeister.org.
0.0.9.3.2.7.e.f.f.f.3.6.6.7.2.e.4.8.0.0.0.3.0.0.0.7.4.0.1.0.0.2.ip6.arpa. 1254 IN PTR panix.netmeister.org.
ptr.dns.netmeister.org. 1265 IN PTR ptr.dns.netmeister.org.
txt.dns.netmeister.org. 592 IN TXT "Format: <text>"
txt.dns.netmeister.org. 592 IN TXT "Descriptive text. Completely overloaded for all sorts of things. RFC1035 (1987)"
naptr.dns.netmeister.org. 1284 IN NAPTR 20 10 "s" "http+N2L+N2C+N2R" "" www.netmeister.org.
naptr.dns.netmeister.org. 1284 IN NAPTR 10 10 "u" "smtp+E2U" "!.*([^.]+[^.]+)$!mailto:postmaster@$1!i" .
mx.dns.netmeister.org. 1292 IN MX 50 panix.netmeister.org.
ds.dns.netmeister.org. 1299 IN DS 56393 13 2 BD36DD608262A026083721FA19E2F7B474F531BB3179CC00A0C38FF0 0CA11657
rrsig.dns.netmeister.org. 1308 IN RRSIG TXT 13 4 3600 20210902104320 20210819095136 56039 dns.netmeister.org. 6VhgJLJXR8f8VUPyC0v5EcS17Z/TfDIdVF7f8L9SceRvSfqz5NMzVD3E aMGN7DaEezXdBhqGaf/KFZxKufmIvA==
rrsig.dns.netmeister.org. 1308 IN RRSIG NSEC 13 4 3600 20210908211756 20210825205219 56039 dns.netmeister.org. 59eCCJ//Y1O69REgNOry9WB+XKNGMATN6EJ6mqtSqj8H31gKI8Dm2+pj FKHVfj5dJV62LVI9tPQHYSJVb/evdA==
dnskey.dns.netmeister.org. 1317 IN DNSKEY 257 3 13 XEn4q8CbG2a4Hw47Ih244BDkwY1tOuprXWKEzMyLPtjO9iIRVt4HLLbx 9YaeaYzRcH91mvCstP8I5liQ0Mn1bA==
ns.dns.netmeister.org. 1324 IN NS panix.netmeister.org.
svcb.dns.netmeister.org. 1343 IN TYPE64 \# 85 2D6E20312070616E69782E6E65746D6569737465722E6F72672E2069 70763668696E743D22323030313A3437303A33303A38343A65323736 3A363366663A666537323A333930302220706F72743D223838383822 0A
https.dns.netmeister.org. 1351 IN TYPE65 \# 25 2D6E2030207777772E6E65746D6569737465722E6F72672E0A
https.dns.netmeister.org. 1351 IN TYPE65 \# 123 2D6E2031202E20616C706E3D2268332C683222206970763668696E74 3D22323030313A3437303A33303A38343A653237363A363366663A66 6537323A333930302220706F72743D22383038302220656368636F6E 6669673D225A57356A636E6C776447566B49474E7361575675644342 6F5A57787362776F3D220A
I guess dnsmasq is somehow deciding to not record the reply and, hence, FTL did not see it, either.
This would explain both the N/A (because the reply type is not available at this time and the 0.0 msec (because this is what will be shown if the reply has not yet been received).
Can you show relevant logs from /var/log/pihole.log ?
My guess is that the log will contain lines like (guessing from the very first screenshot in this discussion)
query[HTTPS] https.dns.netmeister.org from <whereever>
forwarded https.dns.netmeister.org to <somewhere>
validation result is SECURE
BUT no corresponding lines like
reply https.dns.netmeister.org is <xyz>
cached https.dns.netmeister.org is <xyz>
If this turns out to be true, this will be a dnsmasq bug which we need to solve over there. I'm not hesitant to trace this down and submit a patch upstream.
1 Like
Yes, you are right. Except for the TXT query, all other queries without a proper reply type don't have corresponding reply/cached lines.
Aug 26 18:38:13 dnsmasq[24015]: query[A] a.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: cached a.dns.netmeister.org is 166.84.7.99
Aug 26 18:38:14 dnsmasq[24015]: query[AAAA] aaaa.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: cached aaaa.dns.netmeister.org is 2001:470:30:84:e276:63ff:fe72:3900
Aug 26 18:38:14 dnsmasq[24015]: query[SRV] srv.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: cached srv.dns.netmeister.org is <SRV>
Aug 26 18:38:14 dnsmasq[24015]: query[SOA] soa.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: forwarded soa.dns.netmeister.org to 127.0.0.1
Aug 26 18:38:14 dnsmasq[24015]: validation result is SECURE
Aug 26 18:38:14 dnsmasq[24015]: query[PTR] 99.7.84.166.in-addr.arpa from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: cached 166.84.7.99 is panix.netmeister.org
Aug 26 18:38:14 dnsmasq[24015]: query[PTR] 0.0.9.3.2.7.e.f.f.f.3.6.6.7.2.e.4.8.0.0.0.3.0.0.0.7.4.0.1.0.0.2.ip6.arpa from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: cached 2001:470:30:84:e276:63ff:fe72:3900 is panix.netmeister.org
Aug 26 18:38:14 dnsmasq[24015]: query[PTR] ptr.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: forwarded ptr.dns.netmeister.org to 127.0.0.1
Aug 26 18:38:14 dnsmasq[24015]: validation result is SECURE
Aug 26 18:38:14 dnsmasq[24015]: query[TXT] txt.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: forwarded txt.dns.netmeister.org to 127.0.0.1
Aug 26 18:38:14 dnsmasq[24015]: validation result is SECURE
Aug 26 18:38:14 dnsmasq[24015]: reply txt.dns.netmeister.org is Format: <text>
Aug 26 18:38:14 dnsmasq[24015]: reply txt.dns.netmeister.org is Descriptive text. Completely overloaded for all sorts of things. RFC1035 (1987)
Aug 26 18:38:14 dnsmasq[24015]: query[NAPTR] naptr.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: forwarded naptr.dns.netmeister.org to 127.0.0.1
Aug 26 18:38:14 dnsmasq[24015]: validation result is SECURE
Aug 26 18:38:14 dnsmasq[24015]: query[MX] mx.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: forwarded mx.dns.netmeister.org to 127.0.0.1
Aug 26 18:38:14 dnsmasq[24015]: validation result is SECURE
Aug 26 18:38:14 dnsmasq[24015]: query[DS] ds.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: forwarded ds.dns.netmeister.org to 127.0.0.1
Aug 26 18:38:14 dnsmasq[24015]: validation result is SECURE
Aug 26 18:38:14 dnsmasq[24015]: query[RRSIG] rrsig.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: forwarded rrsig.dns.netmeister.org to 127.0.0.1
Aug 26 18:38:14 dnsmasq[24015]: validation result is INSECURE
Aug 26 18:38:14 dnsmasq[24015]: query[DNSKEY] dnskey.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: forwarded dnskey.dns.netmeister.org to 127.0.0.1
Aug 26 18:38:14 dnsmasq[24015]: validation result is SECURE
Aug 26 18:38:14 dnsmasq[24015]: query[NS] ns.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: forwarded ns.dns.netmeister.org to 127.0.0.1
Aug 26 18:38:14 dnsmasq[24015]: validation result is SECURE
Aug 26 18:38:14 dnsmasq[24015]: query[type=64] svcb.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: forwarded svcb.dns.netmeister.org to 127.0.0.1
Aug 26 18:38:14 dnsmasq[24015]: validation result is SECURE
Aug 26 18:38:14 dnsmasq[24015]: query[type=65] https.dns.netmeister.org from 192.168.178.20
Aug 26 18:38:14 dnsmasq[24015]: forwarded https.dns.netmeister.org to 127.0.0.1
Aug 26 18:38:14 dnsmasq[24015]: validation result is SECURE
1 Like
I had some time this morning to check the dnsmasq source and now we know what is happening:
dnsmasq can only cache the following types:
AAAAASRVPTR
plus DNSSEC-relevant stuff.
This is not much but surely the majority of things you will see throughout the day.
The interesting bit is now that dnsmasq only logs "reply abc.com is ..." when caching a reply. It does not log any of the the other types (like SOA, or NAPTR). Just because it doesn't cache them. This also explains why you always see a forwarding. This is what actually happens.
Side note: TXT gets a "reply abc.com is ..." line because there is some special code in another region of the code that prints this into the log file. This code is in a somewhat obscure place and seems replacement-worthy.
My plan is to submit a patch that logs all replies even if they don't enter the cache. Furthermore, I will move the special TXT record handling into the same place.
edit Patch submitted upstream. It proposes logging like
Aug 27 10:33:08 dnsmasq[1497026]: query[SOA] soa.dns.netmeister.org from 127.0.0.1
Aug 27 10:33:08 dnsmasq[1497026]: forwarded soa.dns.netmeister.org to 127.0.0.1
Aug 27 10:33:08 dnsmasq[1497026]: reply soa.dns.netmeister.org is non-cached [SOA]
(emphasis on the last line). This is not everything we need here, however, it is an important step. Whether this is accepted will decide on how we continue here.
Thanks again for bringing this to our attention! I totally missed it and, apparently, many others as well.
3 Likes
Thank you for your dedication, even if the issue lies with dnsmasq rather than Pi-hole.
Few questions though.
Would simply logging the reply already solve the N/A (0.0ms) problem? Even if --log-queries is unset in the config? I guess it would still say N/A for an actual reply, but then you can at least distinguish it from NODATA.
How does it handle NODATA answers, actually? Currently there are specific NODATA-IPv4 and NODATA-IPv6 answers printed to log. Or in case of DNSSEC queries it will simply read no DS. PTR queries are then NXDOMAIN instead of NODATA obviously.
Will the reply line contain the actual answer or does it print literally non-cached [SOA] in your example?
True, although I have seen HTTPS as high as 20% at times. That was my main concern on the original post, I just happened to notice that it also happens for other query types as well.
Caching these other types would be nice, but just logging replies and showing the appropriate reply type in the Query Log is a good first step to gauge if caching would even be useful in a significant way. Caching wouldn't be a priority if 99% of these other types are just NODATA anyway, but right now you can't tell that without manually digging for these records.
No. But this is why I said
FTL gets what it needs by hooking into the log routines of dnsmasq. This has the benefit of least code modifications of the original source code to ensure future updates can be merged without having to resolve merge conflicts all the time. As there is a lot more information available internally than what is written to the log, this is often sufficient (but not always).
However, this has the drawback that FTL does not see stuff for which the log routines are not even called so this is why FTL was still at N/A for such queries (it just never saw any reply at all). Now, as these replies are logged, FTL sees them and can report accordingly.
We have a dedicated reply type called NODATA which is used for a negative response for the domain + type. If the response is negative for the domain overall, we get NXDOMAIN. This can also happen to non-PTR queries, e.g. "dig aggggred.com".
The latter. Otherwise, we'd need to implement how to translate the binary data into human-readable for for each of the query types into dnsmasq. Other tools (such as dig) have this code, however, dnsmasq sees itself just as a DNS forwarder and does simply this: forwarding whatever arrives from upstream as long as certain length checks, etc. are fulfilled. It does not need to understand what it is forwarding. This is actually a good thing because it made HTTPS and other types working even with age-old versions of dnsmasq floating around on the web.
Which brings me to my next question: Do you have Apple devices in your network? If so, how many (about)?
The reason I'm asking is that we've been monitoring HTTPS from the very first moment we've seen reports about it. It is worth mentioning that the specification adding HTTPS and SVCB is still only a draft, i.e., it isn't even fully standardized and, hence, possible subject to change. This is why when we (Pi-hole) chose to adopt this kind of query as HTTPS others (like dnsmasq) hasn't yet implemented that TYPE65 is HTTPS.
You'll see this in pihole.log where such queries are logged as query[type=65] instead of something like query[HTTPS]. I assume it is the same hesitance that causes them to not having implemented caching for this type.
Please check out the custom branch tweak/non_cached_replies using
pihole checkout FTL tweak/non_cached_replies
pihole checkout web tweak/non_cached_replies
This branch is based on the latest beta + my patch sent to dnsmasq upstream + the necessary changes to FTL to make this work.
@Daxtorim It would be great if you could
- run your test script on it and get
BLOBreply type for all of these - check out some
HTTPSqueries (I do not have any Apple devices) and see if they are eitherBLOBorNODATA
edit I just pushed another small update (version vDev-1f18a3e) that fixes a cosmetic issue where the first blocked query after a restart could have been incorrectly shown as allowed. The query was still blocked, only the displayed status was incorrect.
The checkout command is case-sensitive, FTL didn't work, it had to be ftl. Just letting you know.
But trying out the branch, it looks very promising. There are some inconsistencies, however. Maybe they are intentional (you didn't list specific changes).
-
The reply type of the
SRVentry in the Query Log changed fromIPtoBLOB. But given the actual content of the answer,BLOBis more appropriate anyway. -
The log entry for the
SRVquery says<SRV>where as every other entry uses[SRV]format (different brackets;SRVgets cached as opposed to others; I don't care, but consistency). -
The
PTRquery for ptr.dns.netmeister.org doesn't show a reply type. There was a response (the same as my sample output above) but the log entry is still missing a reply line (log attached below). The response time is also missing, but complete absence instead of 0.0 ms is a good thing in my opinion, if there was never a response in the first place. -
The queries for
DNSKEYandDSrecords don't get special treatment with theDNSSECreply type. I assume that is because they weren't part of actual DNSSEC validation and instead just manual lookups, in which case this is fine by me.
This was the second run of the script again, hence the cached status for some queries.
pihole.log
Aug 27 18:02:18 dnsmasq[14628]: query[A] a.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:18 dnsmasq[14628]: cached a.dns.netmeister.org is 166.84.7.99
Aug 27 18:02:18 dnsmasq[14628]: query[AAAA] aaaa.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:18 dnsmasq[14628]: cached aaaa.dns.netmeister.org is 2001:470:30:84:e276:63ff:fe72:3900
Aug 27 18:02:18 dnsmasq[14628]: query[SRV] srv.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: cached srv.dns.netmeister.org is <SRV>
Aug 27 18:02:19 dnsmasq[14628]: query[SOA] soa.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: forwarded soa.dns.netmeister.org to 127.0.0.1
Aug 27 18:02:19 dnsmasq[14628]: validation result is SECURE
Aug 27 18:02:19 dnsmasq[14628]: reply soa.dns.netmeister.org is non-cached [SOA]
Aug 27 18:02:19 dnsmasq[14628]: query[PTR] 99.7.84.166.in-addr.arpa from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: cached 166.84.7.99 is panix.netmeister.org
Aug 27 18:02:19 dnsmasq[14628]: query[PTR] 0.0.9.3.2.7.e.f.f.f.3.6.6.7.2.e.4.8.0.0.0.3.0.0.0.7.4.0.1.0.0.2.ip6.arpa from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: cached 2001:470:30:84:e276:63ff:fe72:3900 is panix.netmeister.org
####### ↓↓↓ This one doesn't have a reply line even though a reply was received ↓↓↓
Aug 27 18:02:19 dnsmasq[14628]: query[PTR] ptr.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: forwarded ptr.dns.netmeister.org to 127.0.0.1
Aug 27 18:02:19 dnsmasq[14628]: validation result is SECURE
#######
Aug 27 18:02:19 dnsmasq[14628]: query[TXT] txt.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: forwarded txt.dns.netmeister.org to 127.0.0.1
Aug 27 18:02:19 dnsmasq[14628]: validation result is SECURE
Aug 27 18:02:19 dnsmasq[14628]: reply txt.dns.netmeister.org is Format: <text>
Aug 27 18:02:19 dnsmasq[14628]: reply txt.dns.netmeister.org is Descriptive text. Completely overloaded for all sorts of things. RFC1035 (1987)
Aug 27 18:02:19 dnsmasq[14628]: query[NAPTR] naptr.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: forwarded naptr.dns.netmeister.org to 127.0.0.1
Aug 27 18:02:19 dnsmasq[14628]: validation result is SECURE
Aug 27 18:02:19 dnsmasq[14628]: reply naptr.dns.netmeister.org is non-cached [NAPTR]
Aug 27 18:02:19 dnsmasq[14628]: reply naptr.dns.netmeister.org is non-cached [NAPTR]
Aug 27 18:02:19 dnsmasq[14628]: query[MX] mx.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: forwarded mx.dns.netmeister.org to 127.0.0.1
Aug 27 18:02:19 dnsmasq[14628]: validation result is SECURE
Aug 27 18:02:19 dnsmasq[14628]: reply mx.dns.netmeister.org is non-cached [MX]
Aug 27 18:02:19 dnsmasq[14628]: query[DS] ds.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: forwarded ds.dns.netmeister.org to 127.0.0.1
Aug 27 18:02:19 dnsmasq[14628]: validation result is SECURE
Aug 27 18:02:19 dnsmasq[14628]: reply ds.dns.netmeister.org is non-cached [DS]
Aug 27 18:02:19 dnsmasq[14628]: query[RRSIG] rrsig.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: forwarded rrsig.dns.netmeister.org to 127.0.0.1
Aug 27 18:02:19 dnsmasq[14628]: validation result is INSECURE
Aug 27 18:02:19 dnsmasq[14628]: reply rrsig.dns.netmeister.org is non-cached [RRSIG]
Aug 27 18:02:19 dnsmasq[14628]: reply rrsig.dns.netmeister.org is non-cached [RRSIG]
Aug 27 18:02:19 dnsmasq[14628]: query[DNSKEY] dnskey.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: forwarded dnskey.dns.netmeister.org to 127.0.0.1
Aug 27 18:02:19 dnsmasq[14628]: validation result is SECURE
Aug 27 18:02:19 dnsmasq[14628]: reply dnskey.dns.netmeister.org is non-cached [DNSKEY]
Aug 27 18:02:19 dnsmasq[14628]: query[NS] ns.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: forwarded ns.dns.netmeister.org to 127.0.0.1
Aug 27 18:02:19 dnsmasq[14628]: validation result is SECURE
Aug 27 18:02:19 dnsmasq[14628]: reply ns.dns.netmeister.org is non-cached [NS]
Aug 27 18:02:19 dnsmasq[14628]: query[type=64] svcb.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: forwarded svcb.dns.netmeister.org to 127.0.0.1
Aug 27 18:02:19 dnsmasq[14628]: validation result is SECURE
Aug 27 18:02:19 dnsmasq[14628]: reply svcb.dns.netmeister.org is non-cached [type=64]
Aug 27 18:02:19 dnsmasq[14628]: query[type=65] https.dns.netmeister.org from 192.168.178.20
Aug 27 18:02:19 dnsmasq[14628]: forwarded https.dns.netmeister.org to 127.0.0.1
Aug 27 18:02:19 dnsmasq[14628]: validation result is SECURE
Aug 27 18:02:19 dnsmasq[14628]: reply https.dns.netmeister.org is non-cached [type=65]
Aug 27 18:02:19 dnsmasq[14628]: reply https.dns.netmeister.org is non-cached [type=65]
As for HTTPS queries specifically, NODATA and NXDOMAIN are also working. You saw BLOB in the other screenshot above. I just used dig TYPE65 domain from my PC, but that shouldn't matter (right?). I'll check for genuine queries when the apple devices come home (they are family's).
edit:
Here are queries from actual iPhones.
CNAME as a reply type also works and blocking as well. (Blocking wasn't broken in the first place, it just happened to be in the shot)
/edit
Right, there isn't a one-size-fits-all approach to parsing all queries regardless of type. Not something I need anyway, was just curious how that was handled.
Yes, two iPhones and an apple watch. Usually the amount of "natural" HTTPS queries hovers between 5% and 10%, but when the apple devices are alone on the network or guests bring additional ones, the amount can be much higher than that. It's rare, hence my "at times", but it does happen.
Thanks you very much for your thorough tests. Once everything settled, I'm looking forward to add all these DNS requests to our CI test to ensure the now improved functionality will be automatically tested in the future.
Yes. Due to the lack of a "one-fits-all" case, everything that wasn't a specific type went into the IP reply bucket. This comes from a very old part of the code and is now better.
Yes, this is something we'd have to change in dnsmasq, too. However, my feeling is that a patch for this would just be ignored because it doesn't actually change anything. This happened to similar patches from others in the past. Let's call it an inconsistency by design.
This is interesting. PTRs can be cached and are handled in yet another region of the code so I missed them. So far, PTR requests coming from local config (incl. /etc/hosts) are logged (can be seen in your screenshot as well) but replies from upstream are not. This will need another dnsmasq patch.
This is intentional.
Yes. Because they are not interpreted by dnsmasq and, hence, they are just pushed through appearing as arbitrary BLOB data. I guess that's fine.
Yes.
Thanks. This reveals something interesting. Blocking is N/A because we respond with REFUSED. This was recommended in the HTTPS draft as reply when the information isn't available. I just checked the most recent draft and this section was removed without replacement. I'll change this to an empty NODATA reply with appropriate logging.
I'll work on the things when the weather gets worse and report back.
After checking the code, I have to slightly correct myself on this statement. "IP address PTRs" are logged and cached, e.g.,
Aug 28 20:42:55 dnsmasq[1570270]: 433 127.0.0.1/46468 query[PTR] 8.8.8.8.in-addr.arpa from 127.0.0.1
Aug 28 20:42:55 dnsmasq[1570270]: 433 127.0.0.1/46468 forwarded 8.8.8.8.in-addr.arpa to 127.0.0.1
Aug 28 20:42:55 dnsmasq[1570270]: 433 127.0.0.1/46468 reply 8.8.8.8 is dns.google
If PTR lookups are not IP-address related (as in your example), they cannot be cached (and, hence, are not logged). I'll work on this tomorrow.
I added recording the non-cachable PTRs and submitted a patch upstream.
Furthermore, I added your script from above - assuming you'd agree - in a slightly modified form (also adding ANY and CNAME) to our automated CI testing so reply types cannot be off in the future without us noticing this:
edit Yet another patch submitted upstream: when the question type is different than the answer type - this isn't cached, either. It happen with CNAMEs or, even more obvious, with ANY queries.
TBH, logging all this stuff makes the corresponding dnsmasq code even more complex than it was before. I rather hope Simon Kelley has a better idea to implement the same. I just didn't want to touch too many places in the code as the bigger the patch, the less likely the merge...
No objections from me.
Now I guess it's a waiting game to see what Simon thinks about it.
Simon applied the series of patches and improved them by doing exactly what I didn't want to do:
and his reply was:
Right. That was a bit of a rabbit-hole.
I started with your patch, and began to make it more elegant by doing
all the logging in extract_address(). In the process I found a
long-standing bug (CNAME chains ending in a PTR record were not handled
correctly) and a couple of problems introduced by the changes.The final outcome is quite a big change, and I did wonder if it would be
better to leave it 'till 2.87, but the long-standing CNAME-PTR bug
tipped the balance in favour of doing it now.I've tagged 2.86rc2. I've systematically tested the affected code, and
we're dog-fooding it now. It would be good to get as much other testing
in as possible before the 2.86.Cheers,
Simon.
Accordingly, I undid my "dnsmasq patch {1,2,3}" commits in this branch and committed his larger patch instead. Everything compiled just fine and the CI did not complain when doing the dig tests. I will admit that I was a bit surprised that everything worked that smoothly.
The code was pushed yesterday evening to this special branch and is now also part of the regular beta branches - to have as many users as possible contribute to testing.
Thanks for your report and assistance, as always 
3 Likes
The new logging works even on the ftl release/v5.9 branch 
, but the web release/v5.6 branch is missing the BLOB reply type.
Thanks for the hint, I've opened a PR awaiting review:
Concerning consistency,
we've got another patch in that makes this use < ... > for types everywhere, even those dnsmasq doesn't know (like Apple's HTTPS):
Sep 2 11:23:52 dnsmasq[1815646]: query[type=65] https.dns.netmeister.org from 127.0.0.1
Sep 2 11:23:52 dnsmasq[1815646]: forwarded https.dns.netmeister.org to 127.0.0.1
Sep 2 11:23:53 dnsmasq[1815646]: validation result is SECURE
Sep 2 11:23:53 dnsmasq[1815646]: reply https.dns.netmeister.org is <type=65>
Sep 2 11:23:53 dnsmasq[1815646]: reply https.dns.netmeister.org is <type=65>
They are now analyzed, too. DNSKEY and DS get DNSSEC type and SECURE status when they have been validated. DS may get NODATA and INSECURE when there is no DS record for the requested domain.
Furthermore, we analyze the DNSSEC status also for cached queries (will be either SECURE or INSECURE as BOGUS queries are not cached).
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.
(I'm aware that this topic is over 1 year old)
Simon Kelley, the maintainer of dnsmasq is currently extending the cache to arbitrary type caching. There are a few minor details we are discussing behind the scenes with him, but this will very likely become available for everyone in the next released FTL version (it is already in the bleeding-edge branch update/dnsmasq).
It'll be an opt-in feature in dnsmasq for backwards-compatibility reasons but I guess Pi-hole will opt-in for you (no promises, pending internal discussions).
I will reopen this topic in case any related comments arrive.
This topic was automatically closed after 7 days. New replies are no longer allowed.