Pihole blocking just about everything now

The issue I am facing:

Pihole seems to block just about everything now days. I can no longer access professional photo services of photos ordered from my schools. If I want to use the googleplay store, that is blocked. If I want to use redflagdeals.ca, almost all the deal links are blocked. In fact many google services and amazon services I have paid for are blocked. It gotten bad enough, I wanted to by a fitness app, and the link to set my payment on google pay was blocked.

My automation is regularly breaking by blocked lookups. And yes, worse of all none of the blocked stuff is logged.

I have been adding one link at a time to my whitelist. But sometimes as in redflagdeals, each time I add a new link, it just goes to a page telling me another link is blocked.

It is bad enough I finally just had to disable pihole completely. I would like to reenable pihole. Up until 5.0 pihole was doing a pretty good job of blocking ads, and only ads. Sure it didn't block all the ads, but it got 90% of of them. Now it seems too extreme.

Is there something I can disable so it becomes more relaxed in blocking again? Or did the advertisers change the way the respond to dns blocks so that pihole is no longer an effective advertisement blocker?

Details about my system:

What I have changed since installing Pi-hole:

If Pi-hole is blocking it, and unless you have elevated privacy settings, Pi-hole logs it.

Please post the token generated by

pihole -d

or do it through the Web interface:

Tools > Generate Debug Log

OK. I found out in the another thread, the logging dialog is a bit confusing. But the blocks are actually getting logged. Setting CNAME_DEEP_INSPECT=false has reduced the blocking back down to previous levels. It looks like there are some incorrect entries in the database.

How do we provide feedback to the actual database of blocked sites?

What database is this? The gravity list is built from your subscribed blocklists. If one of them contains a domain that you feel is blocked in error, provide feedback to the list maintainer. In the meantime whitelist the domain and you will override any blocks for that domain (essentially making it gravity proof)

Yes exactly. How do we determine which list, who is the maintainer. Is this really a 1950's type filtering lists with someone attempting to manually maintain a huge list? You actually need to contact a maintainer?

Yes. See your other thread for an explanation of how to contact list maintainers.

Use pihole -q domain.com or on the Web Interface Tools -> Query Lists to figure out which adlist contains the domain in question.

Basically yes I guess.

Well then, that answers that. It lets me know an huge area where volunteers to try and build a sustainable system for filter list. With individual maintainers, it is easy to maintain the enough control so that advertisers don't just start taking over the block lists to effectively sabotage the pihole. But it also really limits the potential, and can cause the whole thing to end when a key maintainer steps abandons their role, or can no longer dedicate the time needed.

It seems the problem here is there are different types of filtering people need. There basically adblock type filtering. Meaning blocking inline ads from showing in the page. There is cookie type blocking, which means keep advertisers from being able track your activity. Then there is content type blocking. For example, even if I click on it, don't open a page for gambling.

Any URL that ends in with click is implied to be a content type blocking. OpenDNS is evidence that is possible to do at a DNS level. But it is not a one size fits all. Every single user can have a custom list of what they block and allow. And every family will want to block different things. And sometimes we find the categories don't quite mean what we thought it did. For example, maybe we intended to block just porn, but suddenly find any sight talking about breast cancer is also blocked because of breast images. So it needs to be easy to tweak and customize those settings.

Someday, pihole might well be able to handle those categorizations. But for now it is best left to setting a site that does handle it like opendns handle it.

Now if each list has a different maintainer, that means each list owner needs to have a meeting of the minds what they are trying to block, and what they aren't trying to block. If I receive a e-mail from a spammer, it would be tempting to thing I want to block the dns lookups for every single link in their e-mail. And for the most part, if the dns lookups are unique enough, that would actually be appreciate by the average user. But this is going into the realm of content based filtering. The links in that e-mail are not something that are popping up as I browse a webpage. They are links I'm clicking on. Whether it is bait and switch, or I really know what I'm clicking on, I'm taking an action indicating I want to see that content. So blocking outside the context of a content filter is a bad idea.

But here is where it gets worse. As soon as we put CNAME lookups we are suddenly not just blocking that particular spammer's links. Now we are blocking everything pointing to the same service provider. So if I'm hosting my website on freeshell.org, and some other website hosted there starts a spamming campaign, suddenly legitimate access to my website is being blocked. What was probably a mildly in appropriate content filter, has suddenly become a shot gun taking down huge sections of the web is a single blast.

Ideally, the way to solve that is to actually have an indication not just what is being blocked. But why. And what is the appropriate level of blocking. Is this really intended to be shot-gun? Maybe in some cases you need that shot-gun, because the advertisers will regularly rotate the CNAME entries they are using. And if the same advertiser his using thousands of CNAME's the only sane way to block them is with the shot-gun approach.

I would actually find it surprising though if these click* links are examples of that. For example if they are using the Sales Force Marketing campaign extension, they can't just arbitrarily change a link designed to click into their promotion, to a link design to collect tracking data. The API simply won't allow that, unless they have exploited a security hole.

Until there is actually some sort of flag on the gravity records indicating which should be subject to cname's, I would suggest a more useful default is to leave CNAME filtering off. As having normal everyday activities blocked by default will trash the reputation of the project.

The solution to this is simple - whitelest the domain freeshell.org and the domain will not be blocked, regardless of where the CNAME trail leads.

Let's look at a single example. What is a URL that you are trying to visit that is not available due to CNAME blocking?

I don't see the point of providing an explicit example. I already gave one at the start of this discussion. You should be seeing many yourself. If you don't already have a large white list, you either don't browse much, or you already turned off the cname feature.

So yes, I can turn on the cname feature again. Then feed you the next example my wife screams at me about. But frankly, if I keep doing things that disrupts her usage of the internet, she'll just start doing everything over VPN instead, or worse I'll find someone from another ISP arriving at our door because she'll think the problem will be solved by switching to another service provider. As she would prefer the ads over the interruptions and having to call me over for technical support.

For now, I'll just leave CNAME blocking off. As that avoids the issue. And my advice is if make that default for everyone, until this is a more reliable feature.

I have been using Pi-hole V5.0 since before it was in beta form (many months now) and I have had no issues with CNAME blocking causing problems. I have always had it enabled.

In the past 30 days, on the Pi-hole that serves primarily my computer (and sometimes a few of my IOS devices), 1,851 queries have been blocked by a gravity CNAME match. I have whitelisted none of these in response.

On a separate Pi-hole that receives a lot more traffic and serves much of the network including my wife's IOS devices, 466 blocks by gravity CNAME. I whitelisted nothing in response.

One of the other new features of Pi-hole V5 is per client blocking. Put your wife's devices on a separate group, and put limited blocking on that group. Or, apply no blocking at all.

The feature is reliable and works exactly as intended. It was requested by users and implemented as requested. Because it is a new feature, the option is provided to disable it if that feature is not to your liking. I don't see the default becoming CNAME blocking disabled.

Are you referring to https://www.redflagdeals.com/?

I use CNAME blocking and can access all there deals without seeing ads. The only CNAME related blocking on their site is logger.yp.ca (blocked ypg.aws.tagcommander.com).

I suspect you have a lot of domains on your blocklist and maybe some not-so well maintained adlists which trigger a lot more false-positives now with CNAME enabled.

It's reliable.

Edit: Read the Wall Of Text and the issue is that you're not understanding CNAME blocking. And granted it is confusing at first but CNAME blocking works.

CNAMEs:

You tell Pi-hole that you don't want Facebook. We block Facebook. Facebook says "Okay, here's NotFacebook, it's really a CNAME to Facebook but we call it NotFacebook." Pi-hole sees that NotFacebook is CNAME to Facebook and blocks it.

CNAME disabled:

You go to NotFacebook and end up at Facebook anyways, but you think you're at NotFacebook.