Question regarding multiple group assignments for same clients

[tauceti]

Hi,

I am using Pihole 5 and when adding multiple clients e.g.
192.168.0.0/24 and assigning them to groups then what happens if I then add e.g.
192.168.0.33 to other (and/or overlapping) groups?
Which groups will be selected for 192.168.0.3? Only those defined explicitly for that client or also the groups from the 192.168.0.0./24 entry?

Thanks

[jfb]

@tauceti

Which groups will be selected for 192.168.0.3? Only those defined explicitly for that client or also the groups from the 192.168.0.0./24 entry?

Both assignments should apply.

[yubiuser]

At the moment both the first matching will be assigned, but this behavior will change in v5.1

Subnetting is not working with most relevant entry

Problem with Beta 5.0: I have added a subnet entry for a /24 network and assigned it to group id 1. Also I have added a native Client IP-Address with /32 and assigned it to group id 0 and group id 2. The client will get the group id 1. The same behavior when adding the client with the dropdown menu. As I saw, in the selection from the database the first matching record is taken. See line 278-279 in get_client_groupids (gravitiy-db.c). The Select is limited to 1. Maybe there must be evalua…

Always chose best (= maximum match) subnet for clients

[tauceti]

ok thanks understood! So in 5.1 it will use the last added entry then. It would be nice to change the order in that table on the admin page so you can easily adapt it and it will chose lets say the first match found in that list instead of the last added one.
When is 5.1 released?

[yubiuser]

@tauceti

So in 5.1 it will use the last added entry then

It will choose the best fit (eg. /32 over /24) but for same subnet it will prefer the last added one.

@tauceti

When is 5.1 released?

Roughly scheduled for the next weeks.

[Dorkster]

So does that mean if I have my network like 10.0.0.0/16 and I assign the subnet 10.0.2.0/24 to group 'IoT' and than have an IP from the same range explicitly assigned to group 'Xiaomi' only the rules from group 'Xiaomi' will apply?

[yubiuser]

Yes, that's how I think it should work.

[2020-06-09 19:23:23.446 7022] Querying gravity database for client with IP 10.0.10.136...
[2020-06-09 19:23:23.446 7022] SQL: Comparing 10.0.10.136 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-09 19:23:23.446 7022] SQL: Comparing 10.0.10.136 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-09 19:23:23.447 7022] SQL: Comparing 10.0.10.136 vs. 10.0.10.0/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-06-09 19:23:23.447 7022] SQL: Comparing 10.0.10.136 vs. 10.0.10.0/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-06-09 19:23:23.447 7022] SQL: Comparing 10.0.10.136 vs. 10.0.10.136 (subnet 255.255.255.255) - !! MATCH !!
[2020-06-09 19:23:23.447 7022] SQL: Comparing 10.0.10.136 vs. 10.0.10.136 (subnet 255.255.255.255) - !! MATCH !!
[2020-06-09 19:23:23.447 7022] SQL: Comparing 10.0.10.136 vs. 10.0.10.136 (subnet 255.255.255.255) - !! MATCH !!
[2020-06-09 19:23:23.447 7022] --> Found record for 10.0.10.136 in the client (ID 6)
[2020-06-09 19:23:23.447 7022] Querying gravity database for client 10.0.10.136 (getting groups)
[2020-06-09 19:23:23.447 7022] Gravity database: Client 10.0.10.136 found. Using groups [7].

Group ID 7 is associated with /32.

[rossc719]

Hiya.

Just wanted to clarify something, because I think I am seeing something other than what is described here.

I have 192.168.2.0/24 in groups Default and Group1
And 192.168.2.1 is in group Group2

Default has a blacklist entry for foo.com.
Group2 has a whitelist entry for foo.com.

But 192.168.2.1 gets blocked when going to foo.com.

When you say:

Both assignments should apply.

Does this mean that 192.168.2.1 should be a member of all three groups?
Shouldn't the whitelist win?

There is no change in behavior if I change 192.168.2.1 to be in all three groups.
(This is presumably what I would do in v5.1?)

In v5.0 It seems to be acting as though it is finding the 192.168.2.0/24 client mapping first, and stopping there, ignoring the 192.168.2.1 entry completely.

If I add 192.168.2.0/24 to Group2, so that it is a member of all three, then the whitelist takes effect for 192.168.2.1.

Is there a good way to debug what's happening?

[tauceti]

I observe the same behavior. For me it also seems that the /24 assignment which I defined first is used instead of my more specific client IP setting 192.169.0.30 later on.

[yubiuser]

@rossc719

In v5.0 It seems to be acting as though it is finding the 192.168.2.0/24 client mapping first, and stopping there, ignoring the 192.168.2.1 entry completely.

You're right.

Carefully reading the other topic I linked above revealed that only the first matching client mapping applies.

@mgg

As I saw, in the selection from the database the first matching record is taken.
See line 278-279 in get_client_groupids (gravitiy-db.c). The Select is limited to 1.
Maybe there must be evaluated if there are more than 1 matching entry. If there is only one everything works great.

@rossc719

Is there a good way to debug what's happening?

You can add DEBUG_DATABASE=true to /etc/pihole/pihole-FTL.conf and restart pihole. You should see in /var/log/pihole-FTL.log

[2020-06-11 10:05:15.352 28624] SQL: Comparing 127.0.0.1 vs. 10.0.1.0/16 (subnet 255.255.0.0) - NO MATCH
[2020-06-11 10:05:15.352 28624] SQL: Comparing 127.0.0.1 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-06-11 10:05:15.353 28624] Querying gravity database for client 10.0.20.198 (getting best match)
[2020-06-11 10:05:15.353 28624] SQL: Comparing 10.0.20.198 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-11 10:05:15.353 28624] SQL: Comparing 10.0.20.198 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-11 10:05:15.353 28624] SQL: Comparing 10.0.20.198 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-06-11 10:05:15.353 28624] SQL: Comparing 10.0.20.198 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-11 10:05:15.353 28624] Querying gravity database for client 10.0.20.198 (getting groups)
[2020-06-11 10:05:15.354 28624] Querying gravity database for client 10.0.10.182 (getting best match)
[2020-06-11 10:05:15.354 28624] SQL: Comparing 10.0.10.182 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-11 10:05:15.354 28624] SQL: Comparing 10.0.10.182 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-11 10:05:15.354 28624] SQL: Comparing 10.0.10.182 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-06-11 10:05:15.354 28624] SQL: Comparing 10.0.10.182 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-06-11 10:05:15.355 28624] SQL: Comparing 10.0.10.182 vs. 10.0.10.1/24 (subnet 255.255.255.0) - !! MATCH !!
[2020-06-11 10:05:15.355 28624] Querying gravity database for client 10.0.10.182 (getting groups)
[2020-06-11 10:05:15.355 28624] Querying gravity database for client 10.0.1.3 (getting best match)
[2020-06-11 10:05:15.356 28624] SQL: Comparing 10.0.1.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-11 10:05:15.356 28624] SQL: Comparing 10.0.1.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-11 10:05:15.356 28624] SQL: Comparing 10.0.1.3 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-06-11 10:05:15.356 28624] SQL: Comparing 10.0.1.3 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-11 10:05:15.356 28624] Querying gravity database for client 10.0.1.3 (getting groups)
[2020-06-11 10:05:15.357 28624] Querying gravity database for client 10.0.1.4 (getting best match)
[2020-06-11 10:05:15.357 28624] SQL: Comparing 10.0.1.4 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-11 10:05:15.357 28624] SQL: Comparing 10.0.1.4 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-11 10:05:15.357 28624] SQL: Comparing 10.0.1.4 vs. 10.0.10.1/24 (subnet 255.255.255.0) - NO MATCH
[2020-06-11 10:05:15.357 28624] SQL: Comparing 10.0.1.4 vs. 10.0.1.0/16 (subnet 255.255.0.0) - !! MATCH !!
[2020-06-11 10:05:15.357 28624] Querying gravity database for client 10.0.1.4 (getting groups)

Unfortunately it will not tell you which group was assigned in the end. That's why I wrote in the other topic

Subnetting is not working with most relevant entry

While playing with subnet assignment I came across the fact that there is no output to which groups a client belongs if different (non-identical) subnets are defined. I created "subnet clients" /16 and /24 and assigned different groups to them. Connecting a device in both ranges, only the /24 should apply. But there is no indication (on cli and web interface) to which group the client was assigned in the end. [2020-06-09 17:57:29.578 3137] SQL: Comparing 10.0.10.136 vs. 10.0.1.0/16 (subnet 25…

Note: There is an PR which greatly enhances the possible client definitions which has a debug output which client got which group ID. But the commit has not been backported.

@rossc719

Shouldn't the whitelist win?

Yes it will win if it is assigned to the client. That's the reason

@rossc719

If I add 192.168.2.0/24 to Group2, so that it is a member of all three, then the whitelist takes effect for 192.168.2.1.

workes.

As a crude workaround you could try to delete 192.168.2.0/24 as a client and re-add it so that the client ID is greater than for 192.168.2.1 and pihole will assign the right group for it.

If it doesn't work and you don't want to wait for v5.1 you could checkout the development branch of ftl which should fix the subnet assignments. But be aware that this code is still under development und could fail in other circumstances.

[rossc719]

Excellent, thanks!

Now that I understand the behavior of v5.0, I am happy to workaround it until v5.1, which I understand is coming very soon.

Adding 192.168.2.0/24 to Group2 is fine for now. It just ends up whitelisting a site for more clients than I need. But it is relatively harmless.