Invalid ad/domain entry?


#1

So playing with the new feature of removing stuff of your top lists… I have added some entries that get queried for a lot…

I run a domotz pro on my network, and the ttl on this
shard_02-monitoring-us-east-1-cell-1.domotz.com

Is quite low, so lots of queries get generated for it… So its always near the top of the to domains list. So when I go and try and add it to the don’t show list in settings I get this error

Top Domains/Ads entry shard_02-monitoring-us-east-1-cell-1.domotz.com is invalid!
The settings have been reset to their previous values

Clearly this valid… I can query for it, etc. Is there some sort of character limit? I am new to pihole, is there a place to submit bug reports?

Running pihole v2.10 and v2.0 web


#2

The underscore is not recognized as a valid character in DNS, so I believe that’s the reason for your error. Perhaps @Mcat12 @brantje or @DL6ER can confirm.


#3

That is a valid reasoning, other than it resolves. Is it good practice - no. And I am with you it for sure something of contention… In a domain its is is valid.

The problem is it does resolve…

user@ubuntu:~$ dig shard_02-monitoring-us-east-1-cell-1.domotz.com

; <<>> DiG 9.9.5-3ubuntu0.10-Ubuntu <<>> shard_02-monitoring-us-east-1-cell-1.domotz.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 474
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;shard_02-monitoring-us-east-1-cell-1.domotz.com. IN A

;; ANSWER SECTION:
shard_02-monitoring-us-east-1-cell-1.domotz.com. 401 IN A 52.202.223.108

;; Query time: 12 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Fri Dec 23 14:26:07 CST 2016
;; MSG SIZE rcvd: 81

I can reach out to them to remove the _, but since these sorts of fqdn do resolve and work on the public internet - pihole shouldn’t reject putting them in to the system not to show them, etc.

https://www.ietf.org/rfc/rfc2181.txt
The DNS itself places only one restriction on the particular labels that can be used to identify resource records. That one restriction relates to the length of the label and the full name. […] Implementations of the DNS protocols must not place any restrictions on the labels that can be used. In particular, DNS servers must not refuse to serve a zone because it contains labels that might not be acceptable to some DNS client programs.

Now I am with you and do believe rfc https://www.ietf.org/rfc/rfc1034.txt
Puts this limit on host names that are part of a fqdn.

The labels must follow the rules for ARPANET host names. They must
start with a letter, end with a letter or digit, and have as interior
characters only letters, digits, and hyphen. There are also some
restrictions on the length.

So I will reach out to them what looks to be a host in the fqdn having a _

But _ are used in domain names all the time. So to me their validation needs to be loosened… Since clearly the fqdn is resolving and being listed as one of the top queries, I should be able to not list it… Be it the owner is following rfc’s or not - it clearly resolves, and is listed with pihole as top


#4

I’ll change our validation scheme in the future (target release January 2017).


#5

Fantastic, and I also put in a support ticket with the maker of the software domotz, since everything I read _ in the actual host part of the fqdn is not really valid.

While dns rfc’s kind of say hey you should resolve it, be it the host client software likes it or not, etc. But from the rfc’s using _ in host is bad…


#6

@johnpoz


#7

unclear whether it’s pi-hole or me (likely the latter), but i seem unable to blacklist URLs that start with “_LDAP” or “_KERBEROS” even though the domain name (EXAMPLE.COM) has been blacklisted with a wildcard. I have tried “^_” and “^\_” as regex, as well as blocking the specific host name (yep, the entire thing!)… no joy. Is there some debugging that can be turned on to see the results of each regex check to see why it doesn’t seem to be matching the exact hostname, EXAMPLE.COM, and hosts that start with underscore?

UPDATE: found the debug line for pihole-FTL.log (REGEX_DEBUGMODE=true in /etc/pihole/pihole-FTL.conf) and now i see the following lines:

[2019-01-06 14:54:10.858] DEBUG: Regex in line 1 "(^|\.)example\.com$" matches "_kerberos._tcp.usa-phx-033._sites.dc._msdcs.ads.example.com"
[2019-01-06 14:54:10.896] DEBUG: Regex in line 1 "(^|\.)example\.com$" matches "_kerberos._tcp.dc._msdcs.ads.example.com"

it looks like the regex is working but the blacklisting isn’t.


#8

When adding blacklists or wildcard then you type just the plain domain with any subdomains in front. Only regex needs special formed lines.

blacklist

_ldap.example.com 192.168.XX.176 Blocked    (blacklist)

wildcard

_ldap.example.com 192.168.XX.176 Blocked (regex/wildcard)


#9

hmmm. did some more research and it appears that blacklisting is working in some cases: specifically when i use nslookup (both on the same machine and a different machine). Therefore, i’m thinking the DNS query that isn’t getting blocked (evidenced by pihole.log showing forwarded) is somehow different from the query nslookup produces (and the query used by my local web browser which is also blocked).

UPDATE: the queries that are not being blocked appear in pihole.log but not in pihole-FTL.log. when the regex matches the request is blocked; when there is no corresponding entry in pihole-FTL.log the request is not blocked. it seems the requests that aren’t being blocked are SRV requests, not A requests.

I see that dnsmasq can be configured to filter SRV records (by uncommenting #filterwin2k in dnsmasq.conf). However, in the pihole implementation, that means creating a new conf file in /etc/dnsmasq.d (since the existing one is controlled through pihole itself). I made that change, restarted DNS via web admin interface, and now the SRV are being blocked!

the warning in dnsmasq documentation is that disabling SRV may also disable SIP and KERBEROS, so we’ll need to watch for that.

For now, it appears the problem has been solved


#10

And to add the original documentation of the filterwin2k option:

Later versions of windows make periodic DNS requests which don’t get sensible answers from the public DNS and can cause problems by triggering dial-on-demand links. This flag turns on an option to filter such requests. The requests blocked are for records of types SOA and SRV, and type ANY where the requested name has underscores, to catch LDAP requests.

You can add a new file like: 99-extra-settings.conf in /etc/dnsmasq.d/ and avoid conflicting options in other .conf files otherwise Pi-hole won’t start.