# Whitelist always wins

**URL:** <https://discourse.pi-hole.net/t/whitelist-always-wins/60477>\
**Category:** Community Help\
**Created:** [January 15, 2023, 9:31am UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477 "2023-01-15T09:31:56Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![jpgpi250](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/jpgpi250/32/286_2.png) [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)\
**Post date:** [January 15, 2023, 9:31am UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/1 "2023-01-15T09:31:56Z")

</div>

"whitelist always wins" has always been the primary rule of handeling domains, however...

Since one of the last versions ( 109.0.1518.49) of microsoft edge chromium, the browser does a lot of HTTPS queries. I'm blocking these request, have been doing this since this type was first mentioned [here](https://discourse.pi-hole.net/t/local-dns-resolves-first-time-but-not-after-ios-only/41305), since apple has been using these queries for a while now, never had any problems (even an OS upgrade, monterey over big sur, succeeds), blocking these queries. I'm using:

```auto
.*;querytype=HTTPS;reply=nodata

```

since the edge update, I noticed the massive amount of HTTPS queries, no negative impact on the browser experience so far, due to the block.

todays HTTPS count:

 ![image](https://discourse.pi-hole.net/uploads/default/original/3X/2/7/27309974e36f72e76eb6a8cb7a6cd7998a418996.png)

I looked into the data returned by these queries, noticed the returned data can contain a CNAME and / or IP address information.

The problem: I want to block all of the HTTPS queries, as long as there is no negative impact on the browser experience. I don't expect this to happen any time soon, since firefox (possibly other browsers) don't use HTTPS queries at all.

For my banking app (example), I need to whitelist some domains to be able to accept the initial cookie warning and login. Pi-hole sees the whitelist entry, thus forwards the queries for this domain, including the HTTPS queries, which results in the browser getting additional information, that I wanted to block, using the HTTPS query.

This is not a minor problem. To ensure that HTTPS queries do not get an answer, I applied a suricata rule, rejecting all HTTPS queries on the WAN. Despite the Pi-hole regex, I got a lot of reject alerts, I finally figured out this was due to the Pi-hole whitelist entries and had to suppress the alert in suricata (suppress means "applied" but no alert entry logged).

possible solution (@DL6ER feasable?): ~~a domainlist entry type that takes precedence over whitelist entries, e.g. evaluated before whitelist entries are checked?~~ The solution, proposed by yubiuser looks like a better approach, would be great if `querytype=` would allow specifying multiple query types.

---

<div class="post-metadata">

**Author:** ![yubiuser](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/yubiuser/32/12100_2.png) [@yubiuser](https://discourse.pi-hole.net/u/yubiuser)\
**Post date:** [January 15, 2023, 9:58am UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/2 "2023-01-15T09:58:26Z")

</div>

You could try removing the general whitelist entry and add a regex whitelist for that specific domain applying when not query type `HTTPS`

---

<div class="post-metadata">

**Author:** ![jpgpi250](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/jpgpi250/32/286_2.png) [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)\
**Post date:** [January 15, 2023, 10:42am UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/3 "2023-01-15T10:42:53Z")

</div>

> [@yubiuser](#):
>
> You could try removing the general whitelist entry and add a regex whitelist for that specific domain applying when not query type `HTTPS`

this works (example), looks like a good start point for a solution:

```auto
assets.adobedtm.com;querytype=!HTTPS

```

It keeps the option open to whitelist specific HTTPS queries, unfortunately I also want (already do) block all ANY and SVCB queries. It doesn't appear to be possible to specify multiple query types (looked at the [docs](https://docs.pi-hole.net/regex/pi-hole/)). May be a feature that requires a specific syntax, which I don't know.

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 15, 2023, 11:42am UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/4 "2023-01-15T11:42:51Z")

</div>

> [@jpgpi250](#):
>
> possible solution (@DL6ER feasable?): ~~a domainlist entry type that takes precedence over whitelist entries, e.g. evaluated before whitelist entries are checked?~~ The solution, proposed by yubiuser looks like a better approach, would be great if `querytype=` would allow specifying multiple query types.

I just came here in the second you did your edit (I saw the text changing), so that we good timing. It would blow up the regex datastructure quite significantly if we allow multiple query types to be specified, but I will check and come back here.

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 15, 2023, 12:22pm UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/5 "2023-01-15T12:22:22Z")

</div>

Okay, we now have a custom branch

```auto
new/regex_multiple_query_types

```

which supports a comma-separated list of query types, like

```auto
abcabc;querytype=HTTPS,SVCB,TXT

```

Note that combinations with inversions may do strange things, e.g. `!HTTPS,HTTPS` will obviously match all possible query types so is not very useful even if possible. Consider this an alpha-feature, I did some limited but to no means extended testing so any feedback is, as always, appreciated on this.

The CI will need a few moments until the binaries are ready.

---

<div class="post-metadata">

**Author:** ![jpgpi250](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/jpgpi250/32/286_2.png) [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)\
**Post date:** [January 15, 2023, 3:23pm UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/6 "2023-01-15T15:23:56Z")

</div>

> [@DL6ER](#):
>
> `new/regex_multiple_query_types`

the web interface doesn't allow to enter this, so had to edit existing sql entries (and stop start FTL)

It doesn't do what I expected, I entered (Maybe this isn't a correct entry?):

```auto
assets.adobedtm.com;querytype=!ANY,!HTTPS,!SVCB

```

 ![image](https://discourse.pi-hole.net/uploads/default/original/3X/3/2/32a2a021855f4a010bbdf61ff78c70627a77d9f0.png)

type is 2 (regex whitelist, according to [doc](https://docs.pi-hole.net/database/gravity/#domain-tables-domainlist))  
goal is to ~~block all ANY, HTTPS and SVCB queries, allow everything else for this domain.~~ whitelist the domain except when the query type is ANY, HTTPS or SVCB. In that case, the respective blacklist regexes will be applied (these are existing entries):

```auto
.*;querytype=ANY;reply=refused
.*;querytype=HTTPS;reply=nodata
.*;querytype=SVCB;reply=nodata

```

but  
A -\> 0.0.0.0  
ANY -\> valid result (CNAME) -\> no error  
HTTPS -\> valid result (CNAME) -\> no error  
SVCB -\> no result (SVCB record doesn't exist?) -\> no error

also, the FTL log has several entries like, unfortunately, it doesn't indicate which regex the hint is about...:

```auto
[2023-01-15 15:59:45.487 28711/T28727] Hint: This regex matches only specific query types:
[2023-01-15 15:59:45.488 28711/T28727] - AAAA
[2023-01-15 15:59:45.488 28711/T28727] - ANY
[2023-01-15 15:59:45.488 28711/T28727] - SRV
[2023-01-15 15:59:45.489 28711/T28727] - SOA
[2023-01-15 15:59:45.489 28711/T28727] - PTR
[2023-01-15 15:59:45.490 28711/T28727] - TXT
[2023-01-15 15:59:45.490 28711/T28727] - NAPTR
[2023-01-15 15:59:45.490 28711/T28727] - MX
[2023-01-15 15:59:45.490 28711/T28727] - DS
[2023-01-15 15:59:45.491 28711/T28727] - RRSIG
[2023-01-15 15:59:45.491 28711/T28727] - DNSKEY
[2023-01-15 15:59:45.491 28711/T28727] - NS
[2023-01-15 15:59:45.492 28711/T28727] - OTHER
[2023-01-15 15:59:45.492 28711/T28727] - SVCB
[2023-01-15 15:59:45.492 28711/T28727] - HTTPS
[2023-01-15 15:59:45.493 28711/T28727] Hint: This regex matches only specific query types:
[2023-01-15 15:59:45.493 28711/T28727] - ANY
[2023-01-15 15:59:45.494 28711/T28727] Hint: This regex matches only specific query types:
[2023-01-15 15:59:45.494 28711/T28727] - HTTPS
[2023-01-15 15:59:45.496 28711/T28727] Hint: This regex matches only specific query types:
[2023-01-15 15:59:45.497 28711/T28727] - PTR
[2023-01-15 15:59:45.498 28711/T28727] Hint: This regex matches only specific query types:
[2023-01-15 15:59:45.498 28711/T28727] - ANY
[2023-01-15 15:59:45.498 28711/T28727] - HTTPS

```

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 15, 2023, 8:04pm UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/7 "2023-01-15T20:04:33Z")

</div>

> [@jpgpi250](#):
>
> the web interface doesn't allow to enter this, so had to edit existing sql entries (and stop start FTL)

It should be possible, I was able to add them exactly here, too:

 ![Screenshot from 2023-01-15 20-56-49](https://discourse.pi-hole.net/uploads/default/original/3X/7/2/7227fe6fe29cff0981d984d0b53465cb44341fc6.png)

But going the SQL path is obviously okay, too.

> [@jpgpi250](#):
>
> it doesn't indicate which regex the hint is about

`DEBUG_REGEX=true` should be helpful here, this seems to have slipped through into non-debug mode.

> [@jpgpi250](#):
>
> It doesn't do what I expected, I entered (Maybe this isn't a correct entry?):
> 
> `assets.adobedtm.com;querytype=!ANY,!HTTPS,!SVCB`

What do you want this to do? You are saying: Everything except ANY and everything except HTTPS and everything except SVCB. This means: Everything.

I see you want to say: Everything except the three. This isn't possible at the moment but you can go the inverse way and say that something should be active for `A,AAAA,TXT` and whatever you want more.

> [@jpgpi250](#):
>
> In that case, the respective blacklist regexes will be applied (these are existing entries):
> 
> ```auto
> .*;querytype=ANY;reply=refused
> .*;querytype=HTTPS;reply=nodata
> .*;querytype=SVCB;reply=nodata
> 
> ```
> 
> but  
> A -\> 0.0.0.0  
> ANY -\> valid result (CNAME) -\> no error  
> HTTPS -\> valid result (CNAME) -\> no error  
> SVCB -\> no result (SVCB record doesn't exist?) -\> no error

Can you check with `DEBUG_QUERIES=true`, possibly also `DEBUG_REGEX=true` what the lines in `FTL.log` corresponding to the queries you mentioned here are? We're currently busy with testing + preparing the next release of Pi-hole but I should have some more time over the next days.

---

<div class="post-metadata">

**Author:** ![jpgpi250](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/jpgpi250/32/286_2.png) [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)\
**Post date:** [January 15, 2023, 9:07pm UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/8 "2023-01-15T21:07:17Z")

</div>

> [@DL6ER](#):
>
> Can you check with `DEBUG_QUERIES=true`, possibly also `DEBUG_REGEX=true` what the lines in `FTL.log` corresponding to the queries you mentioned here are?

[regex\_multiple\_query\_types.zip](https://discourse.pi-hole.net/uploads/short-url/aoO1qQXzMub9veR7imT4pW7lSyl.zip) (12.4 KB)

> [@DL6ER](#):
>
> I see you want to say: Everything except the three. This isn't possible at the moment

But this is exactly what is needed to ensure everything gets resolved, except the specific query types, as opposed to specifying all the allowed types (which would make a whitelist regex very long, example whitelist regex:

```auto
 ^(.+\.)?(facebook|fb(cdn|sbx)?|tfbnw)\.[^.]+$;querytpe=A,AAAA,SRV,SOA,PTR,TXT,NAPTR,MX,DS,RRSIG,DNSKEY,NS,OTHER

```

- A quick check of the dnsmasq man doesn't reveil any possibilities.
- Is something like `assets.adobedtm.com;querytype=!(ANY,HTTPS,SVCB)` an option?

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 28, 2023, 2:33pm UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/9 "2023-01-28T14:33:37Z")

</div>

Sorry for the long delay, there was so much v6.0 stuff going on, that this slipped my attention in between.

I just pushed a new commit implementing something _like_

> [@jpgpi250](#):
>
> Is something like `assets.adobedtm.com;querytype=!(ANY,HTTPS,SVCB)` an option?

just without the extra `()`. If the first character is an `!`, the list is used as "everything except". If there is _no_ `!` (a simple list like `A,AAAA`), the usual interpretation of "only these" is used. This was considerably easier to implement and is fully backwards compatible with what we have now.

The

> [@jpgpi250](#):
>
> ```auto
> [2023-01-15 15:59:45.487 28711/T28727] Hint: This regex matches only specific query types:
> [2023-01-15 15:59:45.488 28711/T28727] - AAAA
> [2023-01-15 15:59:45.488 28711/T28727] - ANY
> [2023-01-15 15:59:45.488 28711/T28727] - SRV
> [2023-01-15 15:59:45.489 28711/T28727] - SOA
> 
> ```

lines will only be shown in regex debug mode.

I also already added automatic tests for

- `;querytype=ANY,SRV,HTTPS` - matching ONLY `ANY`, `SRV` , and `HTTPS` type queries, and
- `;querytype=!ANY,SRV,HTTPS` - matching everything EXCEPT `ANY`, `SRV` , and `HTTPS` type queries.

Please test away. The binaries already finished compiling while I have been writing this reply.

* * *

**edit** Related pull request (draft):

[https://github.com/pi-hole/FTL/pull/1527](https://github.com/pi-hole/FTL/pull/1527)

---

<div class="post-metadata">

**Author:** ![jpgpi250](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/jpgpi250/32/286_2.png) [@jpgpi250](https://discourse.pi-hole.net/u/jpgpi250)\
**Post date:** [January 29, 2023, 9:53am UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/12 "2023-01-29T09:53:21Z")

</div>

pihole checkout ftl new/regex\_multiple\_query\_types

tried various combinations, see below, player.h-cdn.com is blocked by gravity, www.sevenforums.com isn't (no matches)

changed some entries (enabled/disabled) to check the result, some pihole log examples:

```auto
Jan 29 09:31:56 dnsmasq[13776]: 315 192.168.2.227/56846 query[A] download.microsoft.com from 192.168.2.227
Jan 29 09:31:56 dnsmasq[13776]: 315 192.168.2.227/56846 forwarded download.microsoft.com to 127.10.10.2#5552

Jan 29 09:33:07 dnsmasq[13776]: 326 127.0.0.1/53490 query[HTTPS] download.microsoft.com from 127.0.0.1
Jan 29 09:33:07 dnsmasq[13776]: 326 127.0.0.1/53490 regex blacklisted download.microsoft.com is NODATA

Jan 29 09:33:55 dnsmasq[13776]: 430 127.0.0.1/47141 query[SVCB] download.microsoft.com from 127.0.0.1
Jan 29 09:33:55 dnsmasq[13776]: 430 127.0.0.1/47141 regex blacklisted download.microsoft.com is NODATA

Jan 29 09:34:14 dnsmasq[13776]: 431 127.0.0.1/44947 query[SRV] download.microsoft.com from 127.0.0.1
Jan 29 09:34:14 dnsmasq[13776]: 431 127.0.0.1/44947 forwarded download.microsoft.com to 127.10.10.2#5552

Jan 29 09:33:19 dnsmasq[16039]: 329 127.0.0.1/59917 query[ANY] download.microsoft.com from 127.0.0.1
Jan 29 09:33:19 dnsmasq[16039]: 329 127.0.0.1/59917 regex blacklisted download.microsoft.com is NODATA

Jan 29 09:55:21 dnsmasq[13776]: 805 127.0.0.1/37386 query[A] www.sevenforums.com from 127.0.0.1
Jan 29 09:55:21 dnsmasq[13776]: 805 127.0.0.1/37386 forwarded www.sevenforums.com to 127.10.10.2#5552

Jan 29 09:55:41 dnsmasq[20733]: 807 127.0.0.1/56873 query[ANY] www.sevenforums.com from 127.0.0.1
Jan 29 09:55:41 dnsmasq[20733]: 807 127.0.0.1/56873 regex blacklisted www.sevenforums.com is NODATA

Jan 29 09:56:15 dnsmasq[13776]: 909 127.0.0.1/38449 query[HTTPS] www.sevenforums.com from 127.0.0.1
Jan 29 09:56:15 dnsmasq[13776]: 909 127.0.0.1/38449 regex blacklisted www.sevenforums.com is NODATA

```

 ![image](https://discourse.pi-hole.net/uploads/default/original/3X/4/1/41bff87ca4850907588e728cb12bc21173477361.png)

**everything appears to be working** , it requires a lot of evaluation to verify the answer is correct, for example:  
it may be very confusing why query type A is allowed (using the regex player.h-cdn.com;querytype=!ANY,SVCB,HTTPS) because pihole -q player.h-cdn.com only returns:

```auto
Match found in https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt:
   player.h-cdn.com

```

In order to find the reason why the A query is allowed, you need to run `pihole -q player.h-cdn.com` **AND** `pihole-FTL regex-test player.h-cdn.com`

The solution implies there will be no exact whitelist entries, only regex whitelist entries, when attmpting to block a specific query type.

a whitelist example (domain is blocked due to gravity entry):

- pihole -q ab.tweakers.nl

```auto
 Match found in https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts:
   ab.tweakers.nl

```

 ![image](https://discourse.pi-hole.net/uploads/default/original/3X/5/8/588da77a997907c3e0f40b0defe08a29d22ef4ea.png)

result (blocked A, allowed AAAA, allowed SRV)

```auto
Jan 29 10:42:55 dnsmasq[13776]: 1263 127.0.0.1/60238 query[A] ab.tweakers.nl from 127.0.0.1
Jan 29 10:42:55 dnsmasq[13776]: 1263 127.0.0.1/60238 gravity blocked ab.tweakers.nl is 0.0.0.0

Jan 29 10:43:07 dnsmasq[13776]: 1265 127.0.0.1/50251 query[AAAA] ab.tweakers.nl from 127.0.0.1
Jan 29 10:43:07 dnsmasq[13776]: 1265 127.0.0.1/50251 forwarded ab.tweakers.nl to 127.10.10.2#5552

Jan 29 10:50:30 dnsmasq[13776]: 1301 127.0.0.1/43596 query[SRV] ab.tweakers.nl from 127.0.0.1
Jan 29 10:50:30 dnsmasq[13776]: 1301 127.0.0.1/43596 forwarded ab.tweakers.nl to 127.10.10.2#5552

```

---

<div class="post-metadata">

**Author:** ![DL6ER](https://discourse-cdn.pi-hole.net/user_avatar/discourse.pi-hole.net/dl6er/32/281_2.png) [@DL6ER](https://discourse.pi-hole.net/u/DL6ER)\
**Post date:** [January 29, 2023, 7:14pm UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/13 "2023-01-29T19:14:24Z")

</div>

Thanks for your testing, the PR is now open for review + merge for the next FTL release (which should not take long as `dnsmasq v2.89` is expected soon, too).

> [@jpgpi250](#):
>
> it may be very confusing why query type A is allowed (using the regex player.h-cdn.com;querytype=!ANY,SVCB,HTTPS) because pihole -q player.h-cdn.com only returns:
> 
> ```auto
> Match found in https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt:
> player.h-cdn.com
> 
> ```
> 
> In order to find the reason why the A query is allowed, you need to run `pihole -q player.h-cdn.com` **AND** `pihole-FTL regex-test player.h-cdn.com`

This is going to be improved with v6.0 because the result of `pihole -q` will be provided by FTL itself and not by a `bash` script. FTL knows how to handle the different types and this all will be reflected in the output.

---

<div class="post-metadata">

**Author:** ![system](https://discourse.pi-hole.net/uploads/default/original/3X/7/c/7c8792f649eeb921c5d2b4c41564ffa873d9a2b8.png) [@system](https://discourse.pi-hole.net/u/system)\
**Post date:** [February 19, 2023, 7:14pm UTC](https://discourse.pi-hole.net/t/whitelist-always-wins/60477/14 "2023-02-19T19:14:57Z")

</div>

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.