Slight bug in blacklist uniqueness constraint?

I have added device-metrics-us.amazon.com to my blacklist, and now I'd like to add device-metrics-us-2.amazon.com to my blacklist. However, when I try to do this, I get this error:

**Error, something went wrong!** While executing: **UNIQUE constraint failed: domainlist.domain** Added 0 out of 1 domains

These two URLs resolve to different IP addresses, and they are in fact not duplicates. I can't think of a way that my particular configuration would be responsible for this. And yes, I checked to make sure that the -2.amazon.com URL is not already in the blacklist.

Anyone have any insights into this?

Please post the token generated by

pihole -d

or do it through the Web interface:

Tools > Generate Debug Log

Could successfully add both domains to blacklist.

https://tricorder.pi-hole.net/symdbs4o1z

Oops, my bad. device-metrics-us-2.amazon.com was whitelisted, and there must be a uniqueness constraint that prevents a URL from being both whitelisted and blacklisted? Anyway, solved. Thanks.

It's because they are in the same column domain in table domainlist in gravity. So the unquie constraint applies to all types of manually added domain (black, white, regex).

@DL6ER I wonder if this is a potential bug, as I can imagine users to add a domain they want to be blacklisted (e.g. for default group) and additionally to whitelist for a few users in another group. Maybe the unique constrain should cover domain and type?

Yes, this is a legit use case I haven't thought of. I was more thinking that it will cause confusion if users add the same domain in black- and whitelist and don't see it being blocked. This is exactly the issue the OP had:

so he hasn't seen it.

I agree, @PromoFaux changed the INSERT to a REPLACE, meanwhile. However, we'd still have to replace the gravity table definition by a UNIQUE constraint on two variables:

Question: It we do this, should we then remove the dedicated white- and blacklist pages? This will lead to issues, otherwise.

Yes. This was the plan some time ago, right? You're idea was to keep it for novice users. Removing will inevitable produce some issues and confusion here on discourse but I guess users will get familiar with it.

Ah yes, I was still thinking "small" when changing that part. (as in, one white/blacklist for the entire install). It is indeed a valid possibility that one may wish to have per-group black/whitelist items that contradict each other in the wider sense.

We would need to include a check that whitelisted domain.com and blacklisted domain.com could not both be assigned to the same group.

Well, we did remove the dedicated pages. These black- and whitelist pages there as one and the same as the one under group management , just with some elements hidden.

Why? I thought whitelist proceeds blacklists so there should be no conflict.

I guess it would tell the user "You are already allowing/denying this domain for users of this group. Adding this domain to the opposite list may not have the desired effect"

The conflict my arise in the user's head. That's what I wanted to point at above. They may have it on the whitelist but are not aware of this. The separated white- and blacklist pages may hide this from them when they don't look.

Ok so ideally there would be two warnings.

  1. User adds the domain to a list while it is already on another list. "XYZ is already in your black/whitelist. Do you really want to add it to black/whitelist?"

  2. User adds the domain twice to a group: "You are already allowing/denying this domain for users of this group. Adding this domain to the opposite list may not have the desired effect

Implementing a ping-pong "Click Yes/No" is actually a lot of work. I will do the following:

  • Allow adding of duplicate white-/blacklist entries if there is already an exact opposite already present with membershp in default
  • Generally make the warnings more verbose and explanatory

Though, I think the desired behaviour from the query log should still remain the same. If pressing the blacklist or whitelist buttons there, then it should only be added to the blacklist or whitelist of the default group (and if it already exists on one then continue, as now, to remove if from that list and add it to the most recently requested list)

Adding per-client settings here would maybe start to get too complex...

I absolutely agree.

My head is already messed right now..

PR 1 implemented uniqueness enforced on the pair (domain, type) so you can add domains independently to both:

PR 2 improves the warnings shows to the users:

See descriptions therein.