Whitelist/Groups not respected when using sqlite statements

Expected Behaviour:

Whitelist domains for enabled groups should be respected.

Actual Behaviour:

Sometimes domains are blocked, when I enabled clients group. Otherwise it works at some time.
I use sqlite commands on gravity.db to perform group enabling/disabling.

Debug Token:

https://tricorder.pi-hole.net/jRXbRMk8/

I'm on x64-hardware with official docker image.

Instead of assigning clients to a group with a global regex whitelist entry (as you have done for your groups 4 and 5), why not apply no blocking to that group (as you already have done)?

That will apply no blocking to that group.

For the single client assigned to group 4 (the only active group with this whitelist entry applied), does that client appear in your query log as making any queries from its assigned IP?

I would like to unblock clients temporary without disabling pilhole for the entire network. My idea was to create a group for each client with a global regex assigned. As a result I can enable the group for that client to unblock and disable the group to block ads again.

Sometime the above work, but sometimes there are blocked queries in the query log from that specific client weather the group is enabled or not.

Any other approach to get the disired result is welcome.

You don't need a global whitelist in the group. Applying no adlists or domains to that group is the same as disabling Pi-hole for every client in the group.

If you block nothing in the group, you don't need to whitelist anything in the group.

I see what you're talking about.
But to usually block ad for thise clients/groups I have to assign them to some blocklist.

With your approach I would never assign a blocklist to them and they were unblocked persistantly. Or am I missing something.

What I can try is to turn over the logig and assign the blocklist to that groups. Then it should disable blocking by disabling the group.
Will that work better that a blocklist from default group and a global regex whitelist from another?

What jfb is pointing out is that you can do this without a whitelist regex. Create a group called something like Unrestricted. This group has no adlists, rule or clients assigned to it.

Now go to Clients and find each client that you want to control. For each one, click Add and it will appear in the section below and will show as assigned to the Default group.

To temporarily unblock a client, click its drop-down for Group assignment, click Unrestricted to add it to that group, and click Default to remove it from that group. So you're essentially swapping the ticks over. Because the Unrestricted group has no adlists or rules assigned to it any client who is only in that group has no blocks. When you want to turn blocking back on, just put them back the way they were.

client_groups

Thinking about it, I guess the advantage of a regex to whitelist everything (.*) is that you can assign that rule to the Unrestricted group, and then just toggle that group on or off for chosen clients as needed, without needing to also take them in and out of Default. They can stay in Default all the time, with all its blocks, but while they are also in Unrestricted, everything for them is whitelisted.

I use the first method, but that's because I have some clients which permanently have no restrictions. If I was moving them in and out all the time I would probably use the second method and save a couple of clicks.

Thats exaxtly what I'm trying to do.
But unfortunately queries are block anyway sometimes. And sometimes it works. Same If I turnover the logic and create a group with blocklist for each client and disable that one (instead of enabling a one with whitelist assigned).

OK. I investigated this further. And first I would like to add that I used sqlite statements on gravity.db for enabling/disabling my groups.

I tried all three variation and none of them worked.

  1. Default Blocklist, Enable/disable group with global whitelist
  2. Default Empty group + Block group, Enable/Disable block group
  3. Switching groups from Default blocking to Unrescricted without any blocklist.

Now I found, that all of them work if I do the same via Pihole's web interface instead of a sqlite update statement on gravity.db.
Those statements do enable/disable groups or let my clients switch their group assignments. But groups, whitelists or blacklists (enabled/disbled that way) do not have any effect.

Does anyone know what pihole effectively does, when enabling/disabling a group via webinterface?

When you toggle the list status (enabled/disabled), line 216 executes editAdlist().
Which in turn, sends the updated value to scripts/pi-hole/php/groups.php.

This PHP page executes this SQL command to update the database.

(Full edit_adlist code: web/scripts/pi-hole/php/groups.php at be05b0f61d3fcf796dae4cc3f89f8540b2359325 · pi-hole/web · GitHub)

What were the commands used?

For group enabling/disabling I used this:
UPDATE "group" SET enabled='0' where name='$group'

And for switching groups for clients I used this:
UPDATE client_by_group SET group_id=(SELECT id FROM "group" WHERE name='$group') WHERE client_id=(SELECT id FROM client WHERE comment='$client')"

Is there something else I have to do (from the code), to make things work?

How are you executing this SQL command? Using a shell script?

Are you sure the $group variable is correctly expanded?

Yes, I'm pretty sure as the group is enabled/disabled in webinterface afterwards.

I execute this via docker exec and sqlite3 inside the container.

Those statements seem contradictory?

Or does it mean your SQL statements are working now?

Could you please share the exact statements, exactly as you run them?

Sounds contradictory, but it isn't.
What I'm trying to say is, that sqlite statements work from perspective of what I see in piholes webinterface. After executing them webinterface looks like expected.

But clients do not act like expected. For example:
I switch a client from Default group to Unblock group (with no adlists assigned) with the following bash script and the group assignment in webinterface switches. but I can still see all the blocked URLs in the query log.

client="$1"
group="$2"
docker exec pihole sqlite3 /etc/pihole/gravity.db "UPDATE client_by_group SET group_id=(SELECT id FROM \"group\" WHERE name='$group') WHERE client_id=(SELECT id FROM client WHERE comment='$client')"

Similar behavior when using a statement for enabling/disabling groups.

You are missing a step.
You need pihole restartdns reload-lists to reload lists after changing the database.

This is executed by the PHP code I linked on a previous post:

  1. Line 1060 defines a reload is needed: $reload = true;
  2. And this is executed at the end of the script:

Thanks. Thats what I wanted to know.
Seems that I missed that part in your previous post.

Ended up with this:

client="$1"
group="$2"
docker exec pihole bash -c "sqlite3 /etc/pihole/gravity.db \"UPDATE client_by_group SET group_id=(SELECT id FROM 'group' WHERE name='$group') WHERE client_id=(SELECT id FROM client WHERE comment='$client')\" && pihole restartdns reload-list"

Note, your final line ends

... && pihole restartdns reload-list"

That should be reload-lists, plural

Thanks. It's working now.
On this concern I recognized that pihole restartdns reload-list (wrong singular) does the same than pihole restartdns reload. Maybe the command can be improved,

$ pihole
Usage: pihole [options]
...
 restartdns  Full restart Pi-hole subsystems
               Add 'reload' to update the lists and flush the cache without restarting the DNS server
               Add 'reload-lists' to only update the lists WITHOUT flushing the cache or restarting the DNS server
...