Whitelist appears to be being ignored

I have noticed that my YouTube play history was not saving.
So i double checked that i had whitelisted the recommended domains: s.youtube.com and video-stats.l.google.com
And indeed i do:

pi@pihole:~ $ pihole -w s.youtube.com
  [i] s.youtube.com already exists in whitelist, no need to add!
pi@pihole:~ $ pihole -w video-stats.l.google.com                                  
  [i] video-stats.l.google.com already exists in whitelist, no need to add!

However, looking through my logs i see they are getting blocked:

pi@pihole:~ $ cat /var/log/pihole.log | grep s.youtube.com
Apr 27 00:00:06 dnsmasq[17432]: query[A] s.youtube.com from 192.168.0.67
Apr 27 00:00:06 dnsmasq[17432]: gravity blocked s.youtube.com is 0.0.0.0

Crystal Ball is broken, Magic 8 Ball says "Question Unclear, Ask Again".

Sorry, you're right. I didn't technically ask a question did I :man_facepalming:

I guess I'm asking why is / would a whitelisted domain be blocked?

I'll try removing the whitelist entry and re-adding

Are you using groups? Do you have the whitelist assigned to a group that excludes the client?

A debug token would be really helpful instead of me asking 20 questions and taking 20 posts to work through things. But I think you posted in General because you didn't want to fill out the template and just wanted a quick answer.

Presumptuous, but no, not the case at all, I didn't deem it to be a problem with Pihole itself and so not worthy of being in the help section.

https://tricorder.pi-hole.net/tpsglvigfx

I do use groups, so I fear it may be my setup

The device on which I am testing is a OnePlus mobile

Edit:
I strongly suspect this is down to the way I am using groups

My apologies then for presuming.

Your debug output shows:

*** [ DIAGNOSING ]: Domainlist (0/1 = exact white-/blacklist, 2/3 = regex white-/blacklist)
   id    type  enabled  group_ids     domain                                                                                                date_added           date_modified        comment                                           
   ----  ----  -------  ------------  ----------------------------------------------------------------------------------------------------  -------------------  -------------------  --------------------------------------------------
   43    0           1  0             s.youtube.com                                                                                         2021-04-27 16:22:40  2021-04-27 16:22:40  

So that whitelist (and actually all your whitelists) are applied to group 0, the default group.

The only client that you have assigned to that group is the one with the comment of VPN. (and any client not assigned to a group.)

You have one list assigned to group 0, that is the Unified list. You have other lists assigned to other groups but that Unified list only applies to group 0. The comments for the other lists appears to me that you want the Unified list to apply to EVERY group and then apply the additional lists on top of that Unified list. So Group 5 should have two lists applied. Unified and News. That's not how Pi-hole is working, right now Group 5 has only the single news list applied to it.

That would work if you put a client in Group 0 and Group 5 but it doesn't look like any of your clients are assigned to multiple groups. Example, the tablets in group 10 have only a single list applied to them, and that is the single list you have assigned to group 10.

So, to get where you are going you can do one of two things:

Put the tablets in group 0,4,5,10 and apply the lists from each of those groups to the client

or

Apply your lists to multiple groups. You can change the list that currently is your base list and currently is applied to only group 0 and apply it to all your groups.

No apology needed

I think perhaps I don't fully understand how groups work...the docs aren't particularly clear imho. And I'd consider myself pretty competent.

I'll try and outline what I want, may be better, and to help me wrap why head around it...

I'm using the following lists from Steven black....
Default pihole list (= Unified?)
Unified + Porn (≈ default + porn)
Unified + Fake news
Unified + Fake News + Porn
My understanding was each of these additional lists already included the as default pihole lists. (Unified from Steven Black)

I want the kids tablets to have, let's say full blocking, i.e. default, and fake news and porn.

Whereas other devices, just default, or just fake news.

When we speak about groups, do we refer to groups of devices, or groups of lists?
As I'm sure you can see from the debug, I was working on the premise that groups = lists, hence lists:
Bypass
Porn
Fake news
Fake news + Porn

So, if I understand correctly, the tables should be assigned to group 10 only, which will block all porn, fake news and the default domains.
And if I want to read fake news (hypothetically obvs), another device could have group 5 applied only. Which would block fake news domains + the default list (as this list = unified + fake news)
And the whitelist also needs applying to all groups to allow said domains through....
Which leads me to the obvious question...can this be done in 'bulk' or is it a 1 by 1 job via the admin GUI?

I don't think this is the case. You could check with the list maintainer to confirm.

Clients get assigned to groups, then blocking is assigned to those groups as desired.

Ah, okay. If that's the case then the Adlists assignment is how you want it. I didn't think the lists were inclusive like that. I see from the READMEs that they are Unified Hosts + extensions: hosts/alternates/fakenews/readme.md at 19302c497538d4720936a9bbd7074f4fb3f045ea · StevenBlack/hosts · GitHub

Groups are based off of clients. You create groups to manage how clients domains are resolved:

*** [ DIAGNOSING ]: Groups
   id    enabled  name                                                date_added           date_modified        description                                       
   ----  -------  --------------------------------------------------  -------------------  -------------------  --------------------------------------------------
   0           1  Default                                             2021-03-07 10:22:49  2021-03-07 10:22:49  The default group                                 
   2           1  Bypass                                              2021-03-23 15:56:54  2021-03-23 15:56:54  Bypass pi-hole                                    
   4           1  Porn                                                2021-03-24 15:57:23  2021-04-15 15:15:53  Unified + porn                                    
   5           1  Fake news                                           2021-03-24 15:57:23  2021-04-15 15:16:04  Unified + fake news                               
   10          1  Fake news + porn                                    2021-04-15 15:12:52  2021-04-15 15:16:17  Unified + fake news + porn        

Once you have your groups created you decide what lists should be applied to each group:

*** [ DIAGNOSING ]: Adlists
   id    enabled  group_ids     address                                                                                               date_added           date_modified        comment                                           
   ----  -------  ------------  ----------------------------------------------------------------------------------------------------  -------------------  -------------------  --------------------------------------------------
   1           1  0             https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts                                      2021-03-07 10:22:50  2021-04-15 15:02:07  Unified hosts                                     
   5           1  5             https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews/hosts                  2021-04-15 15:02:38  2021-04-15 15:19:02  Unified hosts + fake news                         
   6           1  4             https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/porn/hosts                      2021-04-15 15:03:15  2021-04-15 15:09:18  Unified hosts + porn                              
   7           1  10            https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-porn/hosts             2021-04-15 15:03:47  2021-04-15 15:19:09  Unified hosts + fake news + porn 

The flexibility of groups means that you can apply lists to multiple groups if you'd like.

Once you have the lists affecting the groups then you assign clients to each group. In your case the tablets go in to group 10 and they have group 10's lists applied.

You can think of a group as being a Virtual Pi-hole or a vLAN. However you can expand on that and have a client in multiple groups if you wanted. If the lists you used were not aggregated and they were unique lists then you'd want the tablets to be members of each group and have all the lists applied. Or you could have the tablets be a single group and then make each list apply to that group. There's a ton of flexibility built in and you can design the scheme in a way that makes sense to you.

I think it's manual. Adding a domain or regex via http://pi.hole/admin/groups-domains.php puts that entry in the default group. You then have to click on Group assignment to map that entry to all the groups that you would like it to be applied.

Just to elaborate on my logic here:

pi@pihole:~ $ pihole -q -all -exact doubleclick.net
 Exact matches for doubleclick.net found in:
  - https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
  - https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews/hosts
  - https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/porn/hosts
  - https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-porn/hosts

Limited testing, but all know block domains I queried appear on each list

Gotcha! Thank you

So I was right, I think :woozy_face:
At least partially

A group can be either adlists, or devices. So technically not 100% wrong.

I think I'll stick with groups = lists, for now, seeing as I seem to understand better now. Much appreciated.

Just need to assign each whitelisted domain to the group's.

A group is a conglomerate of clients, adlists and adlists.

Which docs did you read? I recommend to read the page with the examples (again), which should support your understanding of the group function.

1 Like

All of them, word for word, multiple times.
Maybe my statement was not entirely true. Once it's understood, it's clear. It's just getting there.
In no way whatsoever a criticism, just my own opinion / thoughts / experience

How you use groups is the way that groups work. Use them in the way that gets you what you want accomplished.

Under the hood there are some more details. Looking at the debug run (which you can see in your /var/log directory) you see the columns for the groups tables:

*** [ DIAGNOSING ]: Groups
   id    enabled  name                                                date_added           date_modified        description                                       
   ----  -------  --------------------------------------------------  -------------------  -------------------  --------------------------------------------------

Each group is assigned an unique ID.

*** [ DIAGNOSING ]: Adlists
   id    enabled  group_ids     address                                                                                               date_added           date_modified        comment                                           
   ----  -------  ------------  ----------------------------------------------------------------------------------------------------  -------------------  -------------------  --------------------------------------------------

Adlists are assigned an unique ID but they are applied to group_ids.

*** [ DIAGNOSING ]: Domainlist (0/1 = exact white-/blacklist, 2/3 = regex white-/blacklist)
   id    type  enabled  group_ids     domain                                                                                                date_added           date_modified        comment                                           
   ----  ----  -------  ------------  ----------------------------------------------------------------------------------------------------  -------------------  -------------------  --------------------------------------------------

Domain lists are assigned an unique ID but they also are applied to group_ids.

*** [ DIAGNOSING ]: Clients
   id    group_ids     ip                                                                                                    date_added           date_modified        comment                                           
   ----  ------------  ----------------------------------------------------------------------------------------------------  -------------------  -------------------  --------------------------------------------------

Clients are assigned an unique id but there behavior is affected by the group_ids they are associated with.

Under the hood a group is purely a categorization tool.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.