# RegEx engine improvements
**URL:** https://discourse.pi-hole.net/t/regex-engine-improvements/34751
**Category:** General
**Tags:** regex, v5-2
**Created:** [June 22, 2020, 7:44pm UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751 "2020-06-22T19:44:35Z")
**Posts on this page:** 18
**Page:** 3
### Author:  [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)
#### Post date: [July 8, 2020, 10:11pm UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/51 "2020-07-08T22:11:08Z")
> [@DL6ER](#):
>
> Use a negation for the query type, like:
>
> ```auto
> .*;querytype=!A
>
> ```
appears to be working, test (time) will tell if the device(s) continue to work without problems...
---
### Author:  [@Coro](https://discourse.pi-hole.net/u/Coro)
#### Post date: [July 9, 2020, 5:50am UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/52 "2020-07-09T05:50:51Z")
> [@jpgpi250](#):
>
> test (time) will tell if the device(s) continue to work without problems
Thanks for testing. Just keep in mind that this is about the new regex feature not about experience exchange about how devices behave under this or that DNS protocol surgery. 😉
---
### Author:  [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)
#### Post date: [July 16, 2020, 6:25pm UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/54 "2020-07-16T18:25:45Z")
> [@DL6ER](#):
>
> `.*;querytype=!A`
quick question
Do the branches `new/tre-regex` and `new/cname_inspection_logging` contain all the functionality that pihole v5.1.1 offers?
I installed pihole v5.1.1, but now want to continue testing `.*;querytype=!A`
So far, it has been tested successfully on chromecast, ps4 and a windows 7 computer that can only access other local devices (no internet connections allowed, blocked on firewall). I'm expecting to have to whitelist PTR queries, some connections on the win 7 to other local devices fail at the first try.
---
### Author:  [@PromoFaux](https://discourse.pi-hole.net/u/PromoFaux)
#### Post date: [July 16, 2020, 8:14pm UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/55 "2020-07-16T20:14:28Z")
Short answer: "no"
Longer answer:
[`new/tre-regex` compared with `release/v5.1`](https://github.com/pi-hole/FTL/compare/new/tre-regex...release/v5.1)
[`new/cname_inspection_logging` compared with `release/v5.1`](https://github.com/pi-hole/FTL/compare/new/cname_inspection_logging...release/v5.1)
FTL 5.1.1 has not been released (yet), only the core is at 5.1.1
---
### Author:  [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)
#### Post date: [July 17, 2020, 11:40am UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/57 "2020-07-17T11:40:57Z")
> [@jpgpi250](#):
>
> Do the branches `new/tre-regex` and `new/cname_inspection_logging` contain all the functionality that pihole v5.1.1 offers?
> [@PromoFaux](#):
>
> Short answer: "no"
> [@PromoFaux](#):
>
> FTL 5.1.1 has not been released (yet)
This summarizes everything. FTL v5.1.1 will come sooner than later (maybe even today, maybe Monday, who knows) and once it is out, I will merge everything into the various feature branches floating around. And, finally, they will be marked as ready for merging into `development` a few days later. Even if they contain some subtle bugs we haven't seen so far, it is more easy to have everything in one place and fix it there. We need to test their (possible) interaction anyway at some point.
---
### Author:  [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)
#### Post date: [July 17, 2020, 12:36pm UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/59 "2020-07-17T12:36:53Z")
Thanks, I'll resume testing, once the new dev branch is up to date, which will allow me to test both `new/tre-regex` and `new/cname_inspection_logging`
give us (all testers) a signal when this is done please.
---
### Author:  [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)
#### Post date: [July 29, 2020, 10:48am UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/61 "2020-07-29T10:48:27Z")
Update:
- `new/tre-regex` has been updated to the latest version of the code (`release/v5.2`)
- `new/cname_inspection_logging` is already _included_ in here through the merged PR
[https://github.com/pi-hole/FTL/pull/832](https://github.com/pi-hole/FTL/pull/832)
---
### Author:  [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)
#### Post date: [July 29, 2020, 4:17pm UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/62 "2020-07-29T16:17:23Z")
> [@DL6ER](#):
>
> `new/tre-regex` has been updated to the latest version of the code ( `release/v5.2` )
thanks for the heads up, restarted testing, looks OK
and CNAME logging...
```auto
Jul 29 18:15:25 dnsmasq[25804]: query[A] fonts.gstatic.com from 192.168.2.228
Jul 29 18:15:25 dnsmasq[25804]: forwarded fonts.gstatic.com to fdaa:bbcc:ddee:2::5552
Jul 29 18:15:25 dnsmasq[25804]: reply fonts.gstatic.com is
Jul 29 18:15:25 dnsmasq[25804]: reply gstaticadssl.l.google.com is blocked during CNAME inspection
```
`edit`
As expected (see earlier), need to allow PTR queries: `.*;querytype=PTR`
`/edit`
---
### Author:  [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)
#### Post date: [August 9, 2020, 8:36am UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/63 "2020-08-09T08:36:09Z")
> [@jpgpi250](#):
>
> `edit`
> As expected (see earlier), need to allow PTR queries: `.*;querytype=PTR`
> `/edit`
Ah, right. So it would be a good idea to print a message to the Pi-hole diagnostics are when we compile (at least one) regex that could block PTR requests for (at least) one client.
**edit** Added a warning
---
### Author:  [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)
#### Post date: [August 9, 2020, 10:01am UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/64 "2020-08-09T10:01:31Z")
I assume you mean this:
```auto
[2020-08-09 11:43:34.781 9052M] REGEX WARNING: Invalid regex blacklist filter ".*;querytype=!A": This regex may cause name resolution issues (blocking PTR requests)
[2020-08-09 11:43:34.806 9052M] REGEX WARNING: Invalid regex whitelist filter ".*;querytype=PTR": This regex may cause name resolution issues (blocking PTR requests)
```
I've tested immediately,
blacklist regex:` .*;querytype=!A`
whitelist regex: `.*;querytype=PTR`
the messages are somewhat confusing:
- blacklist message: regex isn't invalid, regex might cause network problems...
- whitelist message: regex isn't invalid, it's a whitelist, so it solves problems...
These warnings are also visible on the tools / Pi-hole diagnosis screen:
```auto
Encountered an error when processing blacklist regex filter with ID 0:
```
It's NOT an ERROR, the regex compiles OK, it should be a WARNING...
I know, all of this is cosmetic, everything works as expected, but it may cause other users to report a problem...
edit
the ID on the Pi-hole diagnosis screen is wrong
/edit
---
### Author:  [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)
#### Post date: [August 9, 2020, 8:40pm UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/65 "2020-08-09T20:40:10Z")
Hmm, yes, maybe it would be better to not include the warning at all. When regex are compiled, we cannot know if there is another regex coming later which is "fixing" this. I also thought about that users may intentionally want to do this (block PTR requests) and they don't want to see such a warning.
Probably better to undo this, there is also the option `;invert` which is not really covered here.
As always, thanks for testing, I'm more and more feeling we should revert this. The complexity of getting all the possible negations does not seem to be justified given that this is unproblematic - users will still see the regex link showing them why the PTRs have been blocked.
---
### Author:  [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)
#### Post date: [August 10, 2020, 7:45am UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/66 "2020-08-10T07:45:54Z")
Now that pihole 5.1.2 (and pihole-FTL 5.2) has been released, I'm again wondering if the **new/tre-regex** branch has everything on board, that v5.2 has.
Please let us know if (and when) the branch is updated to v5.2 + regex engine improvements, this to allow us to continue testing.
The last update of the branch did not only upgrade FTL but also made core and web changes, a heads up, if this is going to happen would be appreciated.
Thanks.
---
### Author:  [@Coro](https://discourse.pi-hole.net/u/Coro)
#### Post date: [August 10, 2020, 8:54am UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/67 "2020-08-10T08:54:42Z")
> [@jpgpi250](#):
>
> Now that pihole 5.1.2 (and pihole-FTL 5.2) has been released, I'm again wondering if the **new/tre-regex** branch has everything on board, that v5.2 has.
Yes, there were no changes to FTL v5.2, it was sitting still for quite some time before being released. So this branch is already up-to-date.
> [@jpgpi250](#):
>
> The last update of the branch did not only upgrade FTL but also made core and web changes, a heads up, if this is going to happen would be appreciated.
It picked up the updates for the other branches. There are no changes in core/AdminLTE related to this.
I assume they will soon mark the regex and client-recognition improvements (MAC, etc.) into `development`. When they have confirmation from us, it may speed this up. I can only say it is working flawlessly for me (both branches). For further changes there is no need to delay this further. Even when I think this is done, any more changes can easily be done separately into `development`. This will even be better in terms of being reviewable.
---
### Author:  [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)
#### Post date: [August 10, 2020, 7:50pm UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/68 "2020-08-10T19:50:34Z")
Yes, this is correct.
---
### Author:  [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)
#### Post date: [August 11, 2020, 5:07am UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/69 "2020-08-11T05:07:45Z")
> [@jpgpi250](#):
>
> Please let us know if (and when) the branch is updated to v5.2 + regex engine improvements, this to allow us to continue testing.
The `new/tre-regex` branch has now been merged into `development` and will not receive any further updates. Please change to `development` using
```auto
pihole checkout dev
```
(this will also change web and core to `development`, but this is how this branch is expected to work, no issues expected, so far)
* * *
Something else: Due to the merge into `development`, the branch `new/mac_clients` picked up the `new/tre-regex` changes as well now. So if you change to that branch (`new/mac_clients`), you can already test **both changes in combination** (plus also the ECS (EDNS Client Subnet) identification if you can use that). I had to fix quite a number of merge conflicts, however, our automated test suite suggests that everything works well.
---
### Author:  [@ryrun](https://discourse.pi-hole.net/u/ryrun)
#### Post date: [March 23, 2021, 8:01pm UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/70 "2021-03-23T20:01:59Z")
Is it possible to inverse a regex? So `\.de$` as inversed regex in the blacklist will only allow ".de" domains.
I tried something like this, but it doesn't work: `\.((?!(com|de|org|net|ms|to|eu|at)).)*$`
---
### Author:  [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)
#### Post date: [March 23, 2021, 8:04pm UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/71 "2021-03-23T20:04:45Z")
> [@ryrun](#):
>
> Is it possible to inverse a regex?
Yes.
```plaintext
\.de$;invert
```
---
### Author:  [@ryrun](https://discourse.pi-hole.net/u/ryrun)
#### Post date: [March 23, 2021, 8:09pm UTC](https://discourse.pi-hole.net/t/regex-engine-improvements/34751/72 "2021-03-23T20:09:15Z")
Thank you, works great! I missed this in the docu. 😉
[Previous page](https://discourse.pi-hole.net/t/regex-engine-improvements/34751.md?page=2)