YouTube plays ads on my nVidia Shield, like Pi-Hole never existed

[jfb]

@PA3DSS

you need uBlock Origin extension

That will work for a browser, but the OP is having problems with a separate device that isn't serving content through a browser.

[keem85]

Hi guys. I checked now, it seems that it's working.. the .128 is my nVidia Shield IP that i statically set. See attached screen

[jfb]

OK. Now that Pi-Hole is seeing the nVidia box, you might still get some ads on various platforms (YouTube is one).

Pi-Hole is a domain blocker, and typically ads have been served from separate domains than the content. By identifying and blocking the ad-serving domains, you can block the ads. Your subscribed blocklists are lists of ad-serving (or malware or phishing, etc) domains.

What YouTube and others are starting to do is serve the ads from the same domain as the content, but using a rotating subdomain. Pi-Hole can't block the ad domains without also blocking the content. It's a never ending cat and mouse game.

Note that even if you are unable to block the YouTube ads completely, your Pi-Hole is still blocking a lot of other content (tracking, ads, etc.) for other domains (Neflix, etc) on the nVidia.

Here are some related threads (long in the first case). I recommend reading them all for some insight.

How do I block ads on YouTube?

indeed, i'm just an idiot and wasn't paying attention. thanks

Youtube ads are not being blocked

Hi! This project looks like interesting. Sadly, youtube ads are not being blocked I've found that there are some domains in /etc/pihole/list.preEventHorizon but it cannot block all of them. Logs are as follows: Jun 10 08:51:12 dnsmasq[21296]: forwarded r5---sn-uxaxjvh5gbxoupo5-x1x6.googlevideo.com to 200.45.191.35 Jun 10 08:51:12 dnsmasq[21296]: reply r5---sn-uxaxjvh5gbxoupo5-x1x6.googlevideo.com is <CNAME> Jun 10 08:51:12 dnsmasq[21296]: reply r5.sn-uxaxjvh5gbxoupo5-x1x6.googlevideo.c…

https://www.reddit.com/r/pihole/comments/8hetls/a_question_and_my_youtube_and_spotify_blocklists/

[keem85]

Ahh I understand! I also found some lists to try to block out as much as possible. Thanks for your detailed feedback. At least now I know that the pihole works.

[keem85]

Should I use blocklist or blacklist when finding .txt lists?

[jfb]

I don't understand this question. Please clarify.

[keem85]

For example I found this: pi-hole/hosts at master · hulkhaugen/pi-hole · GitHub

Some guides says I should paste the whole link in to blocklist under "settings". But I also see a "blacklist" above settings.. So there's a blocklist and a blacklist.. Not sure which one to use..

Also, is there an easy way to catch the ads-addresses? For example opening YouTube on my computer and have an extension of some sort that will catch all the ads addresses for me to go through afterwards?

[keem85]

Blocklist or blacklist.. Which one of them should I use? I need to have a little more detailed description than that msatter. Also, I don't use ublock since I am running a shield.. But I am looking for a way to snap up all the ads addresses.

[jfb]

To clarify the terminology:

1. Block lists are publicly maintained lists of domains, hosted on a web location somewhere. When you subscribe to a "block list", Pi-Hole will go to the web and read the entire contents of that block list and add it to gravity (the master list of blocked domains). If you look in your block lists (Web admin GUI > Settings > Blocklists), you see that it consists of web addresses that lead to block lists maintained at each web site.

I would avoid a lot of the lists you find sprinkled around the internet. Many have been adapted from other ad blocker software (AdBlock, for example) and don't follow the correct format. The best collection of blocklists in my opinion is maintained at: https://firebog.net by wally3k, one of the developers.

2. The black list is a locally maintained (on your Pi-Hole) list of domains that you want blocked, typically put here because they aren't on one of your subscribed block lists.

3. White list is a black list in reverse. It's local, but you are telling Pi-Hole not to block that domain.

My advice - start with the seven standard block lists that you installed with Pi-Hole. See how that works for you. If you are seeing things you don't want, then blacklist them. If things are blocked that you want, white list them.

There is a big temptation to go out and collect block lists like treasure coins. Typically all this will do is give you a lot of false blocks and drive you nuts.

@keem85

have an extension of some sort that will catch all the ads addresses for me to go through afterwards

DNSThingy is a Chrome browser extension that works well to show you all the domains that are requested with a website. It is discussed a bit in this thread, which is worth a read:

How do I determine what domain an ad is coming from?

Block ads or fix broken sites This FAQ is written in the context of finding out how to block an ad that isn't blocked automatically, but the same steps can be used to troubleshoot issues with sites that don't function properly. An example is that sometimes certain payment gateways try to collect metrics on the sale, but these domains are blocked by Pi-hole, so the payment cannot be processed. Also, some links send you through several referrers and if one of them is blocked, you may land on a …

[Jeroen1]

OK. Do not have a nvidia shield, In this device: is it possible to use a proxy?..
If not, blocking the ads will be possible only on DNS level (pihole)
If you can set a proxy, there is options.

I use privoxy and inject the adblock pro rules (easylist) in there.
msatter is promoting ublock, but you can not use ublock on your nvidia shield, unless there is some way of installing plugins on that nvidia shield.

Regarding the blacklists: here is what I am using:

https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
http://sysctl.org/cameleon/hosts
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
http://www.squidblacklist.org/downloads/dg-ads.acl
https://smokingwheels.github.io/Pi-hole/allhosts
http://www.malwaredomainlist.com/hostslist/hosts.txt
http://osint.bambenekconsulting.com/feeds/c2-dommasterlist.txt
http://malc0de.com/bl/ZONES
https://isc.sans.edu/feeds/suspiciousdomains_High.txt
http://osint.bambenekconsulting.com/feeds/dga-feed-high.csv
https://hosts-file.net/grm.txt
https://reddestdream.github.io/Projects/MinimalHosts/etc/MinimalHostsBlocker/minimalhosts
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/KADhosts/hosts
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.Spam/hosts
https://v.firebog.net/hosts/static/w3kbl.txt
https://adaway.org/hosts.txt
https://v.firebog.net/hosts/AdguardDNS.txt
https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt
https://v.firebog.net/hosts/Easylist.txt
https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts;showintro=0
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/UncheckyAds/hosts
https://v.firebog.net/hosts/Easyprivacy.txt
https://v.firebog.net/hosts/Prigent-Ads.txt
https://raw.githubusercontent.com/quidsup/notrack/master/trackers.txt
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.2o7Net/hosts
https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt
https://hosts-file.net/exp.txt
https://hosts-file.net/emd.txt
https://hosts-file.net/psh.txt
https://mirror.cedia.org.ec/malwaredomains/immortal_domains.txt
https://www.malwaredomainlist.com/hostslist/hosts.txt
https://bitbucket.org/ethanr/dns-blacklists/raw/8575c9f96e5b4a1308f2f12394abd86d0927a4a0/bad_lists/Mandiant_APT1_Report_Appendix_D.txt
https://v.firebog.net/hosts/Prigent-Malware.txt
https://v.firebog.net/hosts/Prigent-Phishing.txt
https://raw.githubusercontent.com/quidsup/notrack/master/malicious-sites.txt
https://ransomwaretracker.abuse.ch/downloads/CW_C2_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/LY_C2_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/TC_C2_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/TL_C2_DOMBL.txt
https://v.firebog.net/hosts/Shalla-mal.txt
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.Risk/hosts
https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/annoyances.txt
https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/badware.txt
https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/experimental.txt
https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/filters.txt
https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/privacy.txt
https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/resource-abuse.txt
https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/resources.txt
https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/unbreak.txt
https://raw.githubusercontent.com/EnkoNeko/pie/master/PiHole/Adlists/GoogleAds.txt

Jeroen

[keem85]

Nice, thanks! Will add them.. And yes, I have a proxy setting on the nvidia shield. Is there an wasy way of doing what you're saying?

[Jeroen1]

Please be aware adding these lists might block more than you opted for....you may break some stuff.

@keem85

Is there an wasy way of doing what you’re saying?

Nope
Sorry, Unless you have some exerience with setting up stuff on your raspberry or have exeprience with installing proxy software.
And it is not a wonderpill either, I still have ads sometimes, even with the filters.

You can setup privoxy on the raspberry, but huge adblock list inside privoxy will slow down your browsing as the pi is too slow for this. But it can be done.
If you are up for a challenge: install privoxy on the raspberry pi, get it running on 8118 (default) and set the shield pproxy setting to the ip and port of privoxy..
If you can still do any "network" related on the shield, I can send you the action files needed.

[jfb]

@Jeroen1

Regarding the blacklists: here is what I am using:

Being a stickler on terminology - these are block lists, not blacklists.

[keem85]

But can I put these individually in to the blacklist?

[jfb]

@keem85

But can I put these individually in to the blacklist?

You could open each of those web files, and them put the domains in your blacklist, but this would be the incorrect approach. That would make a static copy of active lists.

By putting those web domains into your block list (subscribing to each block list), when Pi-Hole rebuilds gravity it will pull all the domains from those subscribed block lists into gravity. Additionally, when gravity is rebuilt the duplicates (and there are many) are removed, leaving you with a gravity list of non-duplicates.

The other advantage of subscribing to an active block list (instead of adding lines to your blacklist) is that the block lists on the web are actively maintained (domains are added and removed by the list maintainer). When they change their list, Pi-Hole will detect this on the next gravity update (every Sunday morning via a cron script) and update your gravity list accordingly.

As an example, I copied all the block lists from the post above, added them to a Pi in place of the existing block lists, and rebuilt gravity. This is the end of the pihole -g command output showing about 650,000 duplicated domains in those block lists. These duplicates were removed.

[✓] Consolidating blocklists
[✓] Extracting domains from blocklists
[i] Number of domains being pulled in by gravity: 1942816
[✓] Removing duplicate domains
[i] Number of unique domains trapped in the Event Horizon: 1292304

[jfb]

Here is a good FAQ on block lists. Note that Pi-Hole uses lists formatted in HOSTS format.

How do I add additional block lists to Pi-hole?

Pi-hole's Default Block Lists List Formats It's very important that whatever lists you decide to add to Pi-hole are in HOSTS format. Pi-hole's knows how to parse these lists. As you'll see below, things like Adblock Plus' EasyLists can't be automatically imported into Pi-hole. If you are interested in adding additional domains to block, check out WaLLy3K's blocklist compilation. Easiest way to add additional lists UPDATE: As of version 3.0, adlist management can now be done via the web U…

Many of the lists recommended in the reply above are in other formats, which can cause problems.

If you open https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/annoyances.txt, you will see that the first three lines of the file are as follows and the format of the file is incorrect for Pi-Hole.

! Title: uBlock filters -- Annoyances
! Description: Filters optimized for uBlock Origin, to be used with Fanboy's
!              and/or Adguard's "Annoyances" list(s)

This list is not in text format, it's a csv file: http://osint.bambenekconsulting.com/feeds/dga-feed-high.csv.

Recommendation - use lists that work with Pi-Hole. Wally3K lists meet this criteria.

The Big Blocklist Collection

[Jeroen1]

Apologieis , yes these are BLOCKLISTS👍

[keem85]

Hmm some of the hosts from the last list posted here is blocking my "Sign in" function on YouTube.. Gotta find out which one of them..

[jfb]

That is one of the problems with large numbers of block lists and blocked domains. Many false positives. And several of those lists above block each others' domains.