I just don't get it

[Vyper007]

Guys,

Let me start by saying I "want to love" PiHole, I really do, I think the concept is great, it's an innovative use of the Raspberry Pi and I think the project has real potential.

But this is also where it falls down for me with the product, its always got "real potential" but never seems to quite deliver, for me anyway and I just can't help but wonder if its me just doing something wrong. Let me explain.

PiHole at its core is designed to be at its heart one thing.....A network wide ad blocking DNS proxy, but yet even this one task which the project is designed to deal with, it doesn't seem to ever quite get right. I've scoured the internet trying block list after block list but here's the simple facts of the matter

1. I can't block YouTube ads - Please don't tell me its hard to do and its an ever changing battlefield, I know this yet if I install "Adblock for YouTube" Google Chrome extension that blocks 100% of ads 100% of the time. Why is it a simple Chrome extension does a far better job than an entire software package that is specifically designed to bring the same functionality ?.

2. There's no out of the box DoH or DoT, yeah I'm sure I can bolt on some linux package to do it but why isn't PiHole offering me this, it gives me DNSSEC, yet if I want to secure my DNS requests I simply can't. DNS is at the core of this product why isn't DoT or DoH right there in the GUI ?, surely in this day and age it can't be hard to add with so many DNS services like Google, Cloud Flare, Quad 9 and many others all supporting it.

3. DNS Caching, alright yeah its there, I see it, alive and well, but when I look at just how many of my DNS requests actually get cached its about 5 maybe 10% of the total at best, surely caching should be working a bit harder to cache all these requests and be routinely checking that cache in the background to ensure its up to date. Why isn't the caching ratio much higher than it is. I admit I've just used DNS caching in its default configuration and haven't tried tweaking it but everything I read seems to suggest increasing the cache etc is a bad idea anyway.

And then there's services like ITV Hub, Amazon Alexa (which has now start bolting on ads at the end of the flash briefing news summary), various IOS games etc all continue to spam me with ads and I have a pretty severe block list blocking nearly 2.5 million domains yet these ads seem to walk through my defences with ease.

So whats the deal here ?, why am I still bombarded with ads, I'm more than happy to donate and support future developments but I want to see these issues being actively addressed and I just don't see things getting any better ?. So is it me ?, is it a config issue ?, is there a magic block list out there I just need to add that'll make all my woes disappear ?

Where am I going wrong, I just don't get it ?.

[jfb]

@Vyper007

A network wide ad blocking DNS proxy

Pi-Hole is not a proxy, it a DNS server with filtering capabilities (based on dnsmasq).

@Vyper007

Why is it a simple Chrome extension does a far better job than an entire software package that is specifically designed to bring the same functionality ?

AdBlock for YouTube is a proxy (it inspects the entire URL and content and takes actions based on the inspection). As noted above, Pi-Hole is a domain blocker. If the ads are served from the same domains as the content, it is difficult to block them with a domain blocker.

@Vyper007

There’s no out of the box DoH or DoT

There are open feature requests for these. Cast your vote for the features you want to see included. In the meantime, there are user guides for installing DoH, or even better, a local instance of unbound.

@Vyper007

yet if I want to secure my DNS requests I simply can’t.

What do you believe is required to secure your DNS requests? How do you define "secure"? ISP can't see your DNS traffic, it can't be altered, etc?

@Vyper007

DNS Caching, alright yeah its there, I see it, alive and well, but when I look at just how many of my DNS requests actually get cached its about 5 maybe 10% of the total at best, surely caching should be working a bit harder to cache all these requests and be routinely checking that cache in the background to ensure its up to date. Why isn’t the caching ratio much higher than it is.

DNS requests are cached to the time period specified by the nameserver. If you are using a third party DNS provider as your upstream server, they will generally provide very low TTLs to maintain fresh information. You can override these settings in dnsmasq configuration, but as you noted there are downsides to doing so. You can also install a local instance of unbound, which has a very efficient cache.

@Vyper007

So whats the deal here ? Where am I going wrong, I just don’t get it ?.

You may want to read the Pi-Hole documentation and FAQs on this site to get more familiar with the capabilities and operation of Pi-Hole. If Pi-Hole does not meet your needs, then I would recommend not using it and using a solution that does meet your needs.

[Vyper007]

@jfb

You may want to read the Pi-Hole documentation and FAQs on this site to get more familiar with the capabilities and operation of Pi-Hole. If Pi-Hole does not meet your needs, then I would recommend not using it and using a solution that does meet your needs.

Some fair points there, I respect your point of view and feedback. But ultimately PiHole is an "ad blocking" tool, thats the whole point of it, yet even when I have 2.5 million domains blocked many services seem to sneak through ads right under the radar. Its not just one service, I wouldn't mind if it were just YouTube, but its dozens of them, basically almost anything you care to mention that in someway streams content to you....I dont claim to have the technical know how to maybe appreciate the full scale backend issue here of blocking these ads, but my point is this, if an extension can do it then why can't PiHole, if blocking domains doesn't cut the mustard then surely a rethink is required ?.

[jfb]

@Vyper007

its dozens of them, basically almost anything you care to mention that in someway streams content to you

If you can provide some URL examples from platforms other than YouTube and Facebook, perhaps we can help you determine why you are seeing the ads. Post a URL where you are seeing ads, along with a screen snap of the ads you are seeing.

How do I determine what domain an ad is coming from?

Block ads or fix broken sites This FAQ is written in the context of finding out how to block an ad that isn't blocked automatically, but the same steps can be used to troubleshoot issues with sites that don't function properly. An example is that sometimes certain payment gateways try to collect metrics on the sale, but these domains are blocked by Pi-hole, so the payment cannot be processed. Also, some links send you through several referrers and if one of them is blocked, you may land on a …

[Speedy70]

Hello @ All

I have followed this conversation and would like to say the following.

I think Pi-hole is a brilliant project. The ability to apply filters at the DNS level is an exciting proposition and provides sufficient opportunities to keep advertisements, trackers, spam etc. out of their own network. Of course, there are limits here too - as with many other things. Not every advertisement can simply be filtered out at DNS level. And yes, the advertisers also understand their business and of course try to make life as difficult as possible for DNS filters and add-blockers. Pi-hole is not perfect. But other systems are not perfect either. Browser extensions certainly filter more ads from the browser, but are difficult to install on Smart TV's. I like Pi-hole. It is - for me - just the right helper when it comes to the advertising and tracking delusion of some companies to stop. And it's exciting to experiment with the many filter options the Pi-hole offers. I think everyone should use the tool that brings great benefits for one. For me it's just Pi-Hole.

and ... you can use it .. but it does not have to!

Keep it up!!

[PromoFaux]

First of all, let me start by welcoming you to the forum! We try not to bite too hard, and the community is generally very helpful... So, Welcome!

@Vyper007

ITV Hub

I've had some success with this list for ITV and 4OD, at least on my smart TV. In a web browser there are still ads on 4OD, though I do not mind that so much, I don't watch TV at the computer

https://stopads.io/lists/UKcatchuptvblocklist.txt

@Vyper007

if an extension can do it then why can’t PiHole, if blocking domains doesn’t cut the mustard then surely a rethink is required ?

An extension lives on the device that it is installed on, and monitors the traffic for that device only. Now, whilst it may be possible to proxy all traffic through a Raspberry Pi to monitor it and cut out the ads at source, you're going to end up with suboptimal performance. Suddenly, your entire bandwidth is limited to the throughput capabilities of the device running the proxy. In the case of an Rpi, that will end up being a lot slower than you are used to from your router.

The second issue here, is that the traffic is encrypted in transit. By the time any extension sees the data, it has been unencrypted by the browser (I may be oversimplifying this) and so it can do what it wants. The challenge with having an external device do this, is that it needs to be able to unencrypt the traffic, modify it, and then send it on to the client.

I am not too learned on this, so let the following quote from this blog explain it.

To be able eavesdrop and modify HTTPS communication, mitmproxy pretends to be the server to the client and the client to the server, while positioned in the middle it decodes traffic from both of them. Mitmproxy generates certificates on-the-fly to fool the client into believing that they are communicating with the server. To make the client trust newly forged certificates without raising warnings, it is necessary to manually register mitmproxy as a trusted CA with the device.

As far as I know, nobody on the dev team is comfortable with pushing toward that kind of solution. Suddenly we would be making a tool that allows anyone with access to the Pi-hole device the ability to see all traffic on their network, not just the DNS requests. e.g, Currently, as the administrator of my Pi-hole device at home, I know someone has been to facebook.com because of the DNS lookup, but I don't know what they were up to there. And frankly, it's not something I would want to know, either.

At the end of the day, Pi-hole is not meant to be a one-size-fits-all solution. Neither is it the ad blocker to end all ad blockers. It is one layer, designed to block ads (hell, any request) at the DNS level, and I believe it fulfils that purpose. Sure, ads are going to get through. As mentioned by others there are just some areas in which Pi-hole cannot work. But there are many areas in which Pi-hole can work, that a browser extension cannot. E.g Smart TVs, lightbulbs, fridges, thermostats, cheap IOT things you pickup from gearbest, etc.

You make some valid points, and I hope that I've been able to answer them. It's not so much "You're wrong", but "I get what you're saying, but here are the reasons things are the way they are"

[badnewsblair]

Perhaps a good set of Regex, Wildcards will help catch what's sneaking through?