Automatic parsing of adblock-formatted lists

At the moment we have the ability to match domains (and use a wildcard?).

Also, if we try to add an adblock formatted list such as https://easylist.to/easylist/easylist.txt to the filter lists, gravity spits it out in disgust. unless parsed in a particular way.

Could we please get gravity to automatically parse any such list that it receives and add all the discovered domains (including separators and wildcards) to the block list.

Thanks :smiley:

Take a look here for a script that parses easy list (not tested by the dev team).

Lists that are not in hosts format will typically contain a lot of HTML-specific entries, which Pi-hole doesn't use. The ratio of these elements to the domain names, which we need, is very high. So in the example screenshot below, we just need foxsports.com, not all of the other elements.

In addition, since the domain above may might want to go to, it would be blocked if parsed out, so it may result in a false positive domain that should be blocked.

It's not to stay it can't be done, but the amount of unique domains acquired from such a list would be low and could probably be supplemented with a blacklist or another third party list.

You'r right - you can use the script to parse some of the parts of these lists, but It won't get everything, and it wont be able to use the dividers and wildcards properly.

Plus that particular list that I linked above seems to be the only one that is updated frequently enough to block all of youtube's ads...

Plus it contains thousands of domains and thousands more domains with the special syntax elements (wildcards, dividers etc.)

I think that this is still worth implementing for the sake of compatibility with dozens of existing and frequently updated lists as well as being able to use these special syntax elements for better rule creation...

Hi, new to Pi-hole.

I just installed Pi-hole yesterday for testing and i first did wonder i see ads in YouTube videos and also allot of other ads on websites.
I'm a uBlock Origin user in browsers and AdAway on Android and nowhere i have YouTube ads or other stuff.
I also saw that thread on reddit...

So Pi-hole can't block elements and does not work on iframes and does not do the cosmetic stuff that uBlock does?
So instead of parsing the adblock formatted lists my idea would be doing some script that works together with uBlock.
From looking at the 4 uBlock txt files it seems like they override/remove some rules that are in the EasyList lists?

I'm not a bash or perl guy and don't know how to do it, but maybe:

  1. Download all the files
  2. Remove whats in our whitelist
  3. Apply the uBlock filters
  4. Create a file with the rules that only can be done by uBlock
  5. Create a file that is cleaned up from duplicates and the stuff that is done by uBlock

Then disabling every filter in uBlock and creating one that points to the one that was created by the script on our Pi-hole installation. So all devices have everything and browsers with uBlock can do there cosmetic stuff and have less to do.

1 Like