Thanks for the suggestion, I should of seen this myself, always good to have an expert opinion...
Your latest solution works like a charm, resulting output is perfect,..
Thanks again for your time and effort
New feature: Apply regex only to a specific query type.
Example:
abc;querytype=AAAA
will block
dig AAAA abc
but not
dig A abc
This is still experimental. Tests would be appreciated.
2 Likes
installed FTL (pihole checkout ftl new/tre-regex).
works apparently A and AAAA...
- could you give an example of why/when you want to do this (would be useful)?
- doesn't appear to work for other querytypes (NS)?
- web interface turns (example) (.|^)google.com$;querytype=A into (.|^)google.com$;querytype=a (querytype is always saved in lowercase)
We have seen requests for blocking specific query types. This seems to be the simplest realization. I do not expect extensive usage. You typically want one for all.
True, support for further query types has just been added in
This code has not yet reached the regex branch.
Yes. The query types are intentionally recognized case-insensitive to compensate for this.
I assume you refer to this topic (blocking specific query types).
I have been thinking about this, and found a feature is missing in pihole-FTL, to turn this into a valid business case, I'll try to explain.
I'm already using the database schema, that allows duplicate entries in the domainlist table, so entering an identical whitelist/blacklist is possible. This works great if you want to block something for all clients (default group), but allow access for some clients (example used in earlier conversations: allowfacebook group)
This method cannot be used if a regex like .*; querytype=AAAA is used, because it would result in allowing all AAAA queries for certain clients, when using it as a whitelist regex (whitelist always wins). In order to use the above regex, it needs to be used as a blacklist regex entry, targeting specific clients.
Now comes the dilemma. If I want to apply this regex to all but some clients (use it as a blacklist regex), I need to create a group with all the clients, except the ones I want to allow making AAAA queries. This list (can be) very large, and probably will not be effective (new clients aren't member of this group)
Most firewalls have a solution for this dilemma, simply specify the clients (IPs) you want to be unaffected by the rule, and invert the selection. The result is all clients (IPs) except the ones listed. It looks like this:
and the result is this:
for pihole, this would mean you assign a limited number of clients (IP's) to a group, invert the selection, thus effectively targeting all clients, except the ones listed.
The above regex example would than target the ! AllowAAAAqueries group, making it a lot more effective in an environment where new clients come and go on a regular bases.
Something to consider, while your making all these great changes to pihole-FTL?
Blocking AAAA for all clients and allowing it only for some seems a legit use case (at least theoretically).
You should do the inverse: Create a group with the clients that should not match and add them in there. Then add the new regex only to Default which covers also the new clients.
Even when I tend to disagree on your conclusion above, I do agree that is useful to be able to invert a regular expression altogether. Hence, I added the new keyword ;invert
For instance,
^abc$;querytype=AAAA;invert
will not block abc with type AAAA (but everything else) for the clients attached to it.
This has the same effect as inverting the client selection, however, it is even somewhat more powerful.
something wrong???
I installed the latest pihole-FTL version (pihole checkout ftl new/tre-regex), the client name doesn't appear to be resolved anymore, only the IP is shown...
I haven't touched any code there, maybe you just have to wait a little longer. If they do not appear, check your /var/log/pihole.log if you see according PTR requests with answers.
correct, problem auto corrected
sorry...
I entered the blacklist regex
^abc$;querytype=a;invert
I expected a correct answer for the A record, no answer for AAAA (or anthing else, however, dig results (what am I missing?):
pi@raspberrypi:~ $ dig A abc.com
; <<>> DiG 9.11.5-P4-5.1+deb10u1-Raspbian <<>> A abc.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57238
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;abc.com. IN A
;; ANSWER SECTION:
abc.com. 2 IN A 0.0.0.0
;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Jul 07 23:40:09 CEST 2020
;; MSG SIZE rcvd: 41
pi@raspberrypi:~ $ dig AAAA abc.com
; <<>> DiG 9.11.5-P4-5.1+deb10u1-Raspbian <<>> AAAA abc.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12030
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1472
;; QUESTION SECTION:
;abc.com. IN AAAA
;; AUTHORITY SECTION:
abc.com. 873 IN SOA ns-318.awsdns-39.com. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400
;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Jul 07 23:40:13 CEST 2020
;; MSG SIZE rcvd: 114
If I disable the regex entry, I get all correct answers.
Sorry for waisting your time (you wanted feedback)...
This is awesome, can we have a subcategory for Feature Requests only about additional Regex features like this one? 
Your regex says
however your test is
^abc$ does not match abc.com.
The invert makes it match, however.
So the result you see is expected:
- Block all
Aqueries which match^abc$+ invert = don't blockAqueries which match^abc$(but everything else because this is not-matching withoutinvert!) - Don't do anything for any other type (you requested the regex to be valid only for
Aqueries)
What the?...Sure he wants feedback. Even misunderstandings are important IMO as seeing users having issues understanding things may help writing the documentation in the end.
I assumed the invert option only affected the querytype, but after doing some more tests, I must conclude it works as follows (correct me if I'm wrong).
regex from this topic
".*;querytype=A;invert
- pihole-FTL looks at the result of
pihole-FTL regex-test "google.be" ".*;querytype=A", result:.*;querytype=A matches - pihole-FTL than inverts the result.
There appears to be no way to achieve what I wanted (hoped for), apply all rules (gravity, regex, ...) but only allow A queries for a specific device (with a single regex).
I totally misinterpreted the invert function, my mistake...
I might misunderstood what you want, but blocking all A queries for all except one device should be possible by:
- add
.*;querytype=Ato group default as blacklist regex - add
.*;querytype=Ato group "Allow A" as whitelist regex - assign specific client to group "Allow A"
No, that's NOT what I want.
for some devices, I only want to allow A queries, block all others query types (not sure if the devices will still work correctly, need to be able to setup the rules, in order to test)
- device is in the default group
- block, if gravity entry or regex match for default group
- block if query != A (device also assigned to allowOnlyAqueries group)
It really is a low priority (test). If it isn't possible, so be it...
This is what he said:
but I can see how this can be confusing a bit.
This does not seem possible with a single regex. Is there really a "business case" for this? If so, I they can surely add this as well, otherwise the advise of @yubiuser seems to indeed do what you are looking for.
Blacklist .*, and whitelist .*;querytype=A ?
whitelist always wins, as far as I know, whitelist entries are processed first by pihole-FTL.
.*;querytype=A would almost always result in successful resolution of the DNS query (All A queries are successful).
That is my assumption (test case), I think some devices can work with only results for type A queries, but I still want them to follow the general (default group) rules, e.g. for example ssl.google-analytics.com should remain inaccessible for the device.
OK now I got it.... tricky case