Possible bug when PiHole try to parse downloaded list commented using ! and not with #

SStatuti · July 15, 2025, 12:45pm

Please follow the below template, it will help us to help you!

Hello,
today i updated gravity before update PiHole v6.1.3 but from this point was unable to reach github.com
I found the list who generated the problem and checked inside for a error in list but was correct.

https://raw.githubusercontent.com/gioxx/xfiles/master/upd.txt
block list
added: 2024-04-27 15:10:23
last modified: 2024-04-27 15:10:23
last updated: 2025-07-14 20:41:58 (452.575 domains)
enabled, used in 1 group
no comment
matching entries:
||github.com^

I identified the only 3 points where GitHub.com was mentioned and are:

! Hosting: GitHub.com
! Database: https://github.com/mitchellkrogza/Phishing.Database
||myether[REMOVED]et-github.com^

in gravity.sh row 217 excludes only rows starting with # and not starting with !, i propose to modify this line:

grep -v '^ *#' <"${src}" | while IFS= read -r domain; do
in
grep -v -E '^( *#|!)' <"${src}" | while IFS= read -r domain; do

Expected Behaviour:

Pihole's parser should skip commented rows in list using '!'
example
! Hosting: GitHub.com

Actual Behaviour:

domain isn't skipped and added to gravity

Debug Token:

not necessary

DanSchaper · July 15, 2025, 6:09pm

The bang character is not a character that is generally accepted or viewed as a comment indicator. This isn't a change that I'm willing to make. The list maintainer needs to format their list in a way that is generally acceptable. There really should not be any domains lines that are commented, they should just be removed, that's what git is for. The removal of lines prepended with # is for ignoring frontmatter or headers.

SStatuti · July 15, 2025, 6:36pm

hi Dan, i found this https://help.adblockplus.org/hc/en-us/articles/360062733293-How-to-write-filters#special-comments

and the owner of this list is following these rules, ! is used for adBlock and # for hostFiles (Linux and Windows systems).

inside _https://easylist.to/easylist/easylist.txt i can read:

! Homepage: https://easylist.to/
! Licence: https://easylist.to/pages/licence.html
! GitHub issues: https://github.com/easylist/easylist/issues
! GitHub pull requests: https://github.com/easylist/easylist/pulls

so ! is used for comment.

rdwebdesign · July 15, 2025, 7:11pm

Note:

The list you used as example (https://easylist.to/easylist/easylist.txt) is not really compatible with Pi-hole.

Even removing the "comments" starting with !, almost every entry will be ignored by Pi-hole because they are not domains.

Only lines starting with || and ending with ^ will be used. Everything else will be ignored by Pi-hole:

### These entries are invalid for Pi-hole:

||securecloud-smart.com^$popup  # This rule was made to work only with CSS class "popup", but Pi-hole doesn't see content

||141.98.82.232^  # IPs are not domains

/(https?:\/\/)217\.182\.11\..{100,}/  # Pi-hole doesn't receive queries containing `https://` or IPs 

### and so on...

DanSchaper · July 15, 2025, 7:17pm

The gravity code that you linked to is not involved with the list parsing process anymore, it is all handed off to pihole-FTL at pi-hole/gravity.sh at cef7fd4b02c9ae4314e7478c0d542379bfce7f65 · pi-hole/pi-hole · GitHub

And easylist does parse down to some usable:

Including support for ABP style lists (that was in the original Pi-hole code but removed around version 4 because it was very difficult to try and figure out what the list was intending to do) meant that we only allow and recognize a single format for the domains. I'm not fully sure why ||github.com^ was seen or converted from ! github.com because we only look for that specific ||<domain>^ pattern and do not try to convert or modify any other lines.

That said, I'll check on the FTL side to see how that parsing process is handling things. It may indeed already account for the ! style comments.

DanSchaper · July 15, 2025, 7:44pm

Here's the parsing code and it's pretty well commented. The ABP style comment is skipped so I'm not sure how you ended up with ||github.com^ as a filter.

github.com/pi-hole/FTL

src/tools/gravity-parseList.c

bc185680f


      
          			continue;
          
          		// Remove trailing whitespace
          		while(read > 0 && isspace(line[read-1]))
          			line[--read] = '\0';
          
          		// Skip empty lines
          		if(read < 1)
          			continue;
          
          		// Skip lines having any of the following characters:
          		// ! = ABP-style comment
          		// # = bach-style comment
          		// ; = PHP-style comment
          		// [ = ABP header lines
          		if(line[0] == '!' || line[0] == '#' || line[0] == ';' || line[0] == '[')
          			continue;
          
          		// Remove lines containing ABP extended CSS selectors ("##",
          		// "#$#", "#@#", "#?#") and Adguard JavaScript (#%#)
          		char *hash = strchr(line, '#');

robgill · July 15, 2025, 9:53pm

This isn't a Pi-hole issue.

The blocklist in question previously included the line:
||github.com^

Which, as I write this, was removed 17 hours ago.

Presumably by the time you downloaded the list to see if it contained github.com the correction had already been made, but your gravity hadn't been updated (because it couldn't get to github to get the new list) and was still blocking it.

system · August 5, 2025, 9:54pm

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.