Problem with Beta 5.0:
With the Beta 5.0, using groups, I am able to match on exact domains but not regex domains. For the first client query, the pihole-FTL.log regex_debug has
"Regex blacklist ID 1 not enabled for this client"
Looking through the gravity.db, I see entries matching the expected per the configured web UI with groups, clients, and domains all aligned that I would expect them to match. However, querying for the first time from any client, produces the above rows (sometimes incrementing 1 - 20 completely, sometimes incrementing 1 - 20 odds only).
I also have this issue. Nothing interesting shows up in pihole-FTL.log though. Am I missing some sort of config option? I can give debug token if necessary as well.
[2020-02-01 08:21:33.787 9092] REGEX_IGNORECASE: Disabled. Regex is case sensitive
and
[2020-02-01 08:27:02.784 9094] Regex blacklist ID 1 not enabled for this client
[2020-02-01 08:27:02.784 9094] Regex blacklist ID 2 not enabled for this client
[2020-02-01 08:27:02.784 9094] Regex blacklist ID 4 not enabled for this client
[2020-02-01 08:27:02.784 9094] Regex blacklist ID 5 not enabled for this client
[2020-02-01 08:27:02.784 9094] Regex blacklist ID 7 not enabled for this client
[2020-02-01 08:27:02.785 9094] Regex blacklist ID 8 not enabled for this client
[2020-02-01 08:27:02.785 9094] Regex blacklist ID 10 not enabled for this client
[2020-02-01 08:27:02.785 9094] Regex blacklist ID 11 not enabled for this client
[2020-02-01 08:27:02.785 9094] Regex blacklist ID 13 not enabled for this client
[2020-02-01 08:27:02.785 9094] Regex blacklist ID 14 not enabled for this client
[2020-02-01 08:27:02.785 9094] Regex blacklist ID 16 not enabled for this client
[2020-02-01 08:27:02.785 9094] Regex blacklist ID 17 not enabled for this client
[2020-02-01 08:27:02.785 9094] Regex blacklist ID 19 not enabled for this client
Here are some similar ones that I have in place (note: index 59 was entered through the add (wildcard) blacklist button as a testing entry):
sqlite3 "/etc/pihole/gravity.db" "select * from 'domainlist'"
-snip-
17|3|^(.+[.-])?ad[sxv]?[0-9]*[.-]|1|1579905762|1580062689|Migrated from /etc/pihole/regex.list
18|3|^(.+[.-])?adse?rv(er?|ice)?s?[0-9]*[.-]|1|1579905762|1580062691|Migrated from /etc/pihole/regex.list
19|3|^(.+[.-])?telemetry[.-]|1|1579905762|1580062695|Migrated from /etc/pihole/regex.list
20|3|^(www[0-9].)?xn--|1|1579905762|1580062697|Migrated from /etc/pihole/regex.list
21|3|^adim(age|g)s?[0-9][.-]|1|1579905762|1580062701|Migrated from /etc/pihole/regex.list
22|3|^adtrack(er|ing)?[0-9]*[.-]|1|1579905762|1580062702|Migrated from /etc/pihole/regex.list
23|3|^advert(s|is(ing|ements?))?[0-9][.-]|1|1579905762|1580062703|Migrated from /etc/pihole/regex.list
24|3|^aff(iliat(es?|ion))?[.-]|1|1579905762|1580062704|Migrated from /etc/pihole/regex.list
25|3|^analytics?[.-]|1|1579905762|1580062706|Migrated from /etc/pihole/regex.list
26|3|^banners?[.-]|1|1579905762|1580062707|Migrated from /etc/pihole/regex.list
27|3|^beacons?[0-9][.-]|1|1579905762|1580062708|Migrated from /etc/pihole/regex.list
28|3|^count(ers?)?[0-9]*[.-]|1|1579905762|1580062710|Migrated from /etc/pihole/regex.list
29|3|^mads.|1|1579905762|1580062711|Migrated from /etc/pihole/regex.list
30|3|^pixels?[-.]|1|1579905762|1580062712|Migrated from /etc/pihole/regex.list
31|3|^stat(s|istics)?[0-9][_.-]|1|1579905762|1580062717|Migrated from /etc/pihole/regex.list
32|3|^track(ers?|ing)?[0-9][_.-]|1|1579905762|1580062719|Migrated from /etc/pihole/regex.list
33|3|^traff(ic)?[.-]|1|1579905762|1580062720|Migrated from /etc/pihole/regex.list
-snip-
59|3|(.|^)youtube.com$|1|1580531062|1580531075|
Another point of reference from my testing showing it does seem to match on the regex through the -q argument.
pihole -q ads.com
Match found in regex blacklist
^(.+[.-])?ad[sxv]?[0-9]*[.-]
[i] Over 100 results found for ads.com
This can be overridden using the -all option
pihole -q ads.com 527ms Sat Feb 1 12:06:47 2020
Match found in regex blacklist
^(.+[-_.])??m?ad[sxv]?[0-9]*[-_.]
[i] Over 100 results found for ads.com
This can be overridden using the -all option
@DL6ER Mine is in a different state than when I initially reported. Playing with it a bit, it seems possible to delete all clients and groups, restart services, and get it back to a working state. I am not yet sure what triggers getting in to the bad state.
You have client 192.168.1.5 assigned to GroupA. The regex filters are assigned to group Unassociated. There is no overlap between the client and the regex filters and, hence, they are all disabled for this client.
Either add GroupA to all regex filters you want to be enabled for the client or add Unassociated to the client to have it use all the regex filters.
Could you again describe precisely what is the bad state? Just in case it is not that no regex filter is seen enabled for a client. I want to make sure to catch everything.
The bad state is that regex filters are not being applied to incoming DNS queries regardless of group membership.
At the time of my pictures, I could not replicate the problem because I had purged groups and clients and then rebuilt them. Since then, I think it may be a case where not all regex filters are getting parsed.
If I add a new one on my current list, I am not seeming to get a match against it but some of the older existing ones are still working.
If I take an older one that matches, delete it then add it to the end of the list and restart services, it stops matching.
Let me repeat what I understood just to ensure there are no misunderstandings between the two of us.
Group membership matters. A lot.
Are both, the client and the regex filters belong to the same group?
Example: Client blongs to GroupA. Regex belongs to GroupA as well -> it works.
When you delete the regex and add it anew, it will have the default group assignment to Unassociated. This will never match for the client as there is no group overlap between them.
The problem now is that, even if you add the newly added regex filter to GroupA (as the client you're testing from belongs to), it still doesn't match?
Here is what I did this morning. I am matching against ads.com as a test case using the Regex string:
^(.+[.-])?ad[sxv]?[0-9]*[.-]
This was working as expected for a blacklist regex match for all groups this morning.
tyr is assigned 192.168.1.6, recognized by pihole, and is not in a group so default "unassociated" membership.
Then I deleted the regex through the WebUI and pasted the same regex back in and added it again. I added "GroupA" to have this match again and kept unassociated with the match. (I have also created a third group called "All ads allowed" that has no domains enabled.)
Now tyr can resolve that test domain:
For historical, the same was true for a member of GroupA (blocking as expected then allowing after delete/readd):
Okay, please run the following database queries for me and post the printed output:
sqlite3 /etc/pihole/gravity.db
tyr related infos:
SELECT COUNT(*) FROM client WHERE ip = '192.168.1.6';
SELECT GROUP_CONCAT(group_id) FROM client_by_group WHERE client_id = (SELECT id FROM client WHERE ip = '192.168.1.6');
based on this I can generate a few more requests for you to try against your database.
