Domain address is not allowed on blacklist?

TheEngiNerd · September 13, 2022, 3:52pm

Hello all!

I am manually adding YouTube addresses to the blacklist, to help notch some of the ads. The following address is said to be invalid to PiHole for invalid characters, though it is not clear why.

r2---sn-5ualdnss.googlevideo.com

However, the following address is valid and allowed:

rr3---sn-p5qlsny6.googlevideo.com

My install is Pi-hole 5.12 on Windows10 using the "Pi-Hole for WSL1" install found on Github.

Thanks in advance for any pointers or help!

yubiuser · September 13, 2022, 5:31pm

This does not happen on the current development branch and was likely fixed by Do not apply IDNA conversion to RegEx domains by yubiuser · Pull Request #2336 · pi-hole/AdminLTE · GitHub

We preparing a new release quite soon, so please get back to us if it does not work after the next update.

TheEngiNerd · September 13, 2022, 7:48pm

Thanks for the quick response!! I will keep an eye out for the next release and will report back if the problem still persists.

Thanks!

jfb · September 13, 2022, 11:50pm

You will find this is a fruitless endeavor.

TheEngiNerd · September 14, 2022, 2:04am

Blockquote

You will find this is a fruitless endeavor.

Blockquote

Perhaps , but I am seeing some positive impact already of manually adding ad links to the blacklist. If I can get to give me even 10% fewer political ads, it will be worth it!!

TheEngiNerd · September 16, 2022, 3:45pm

Hi yubiuser ,

I updated my Pi-hole with the latest release, and I am still seeing the issue. However, I did find that I could add the offending address to the 'Adlist group management' list, but I could not add it to the 'Domain management' as a Blacklist entry. Hope this detail will help narrow down the issue.

Pi-hole v5.12.1
FTL v5.18
Web Interface v5.15

Thanks!

jfb · September 16, 2022, 4:31pm

The fix is only in our development branch. This will be included in our next release.

If you would like to see the fix or use it now, you can check out the development branch:

pihole checkout dev

To go back to master branch,

pihole checkout master

yubiuser · September 16, 2022, 4:38pm

I looked at the code again and the reason it fails is

chrko@ThinkPad-X230:~$ dig r2---sn-5ualdnss.googlevideo.com
dig: 'r2---sn-5ualdnss.googlevideo.com' is not a legal IDNA2008 name (string contains forbidden two hyphens pattern), use +noidnin

We convert all domains to punycode because this is how they would be request from clients (think of something like täst.com).

github.com

pi-hole/AdminLTE/blob/4d12ee7f9a5a1353e4a1d6ea21bd4d535ff88715/scripts/pi-hole/php/groups.php#L605


      
          if (!is_string($domain) || strlen($domain) == 0) {
              continue;
          }
          
          
if ($_POST['type'] != '2' && $_POST['type'] != '3') {
              // If not adding a RegEx....
              $input = $domain;
              // Convert domain name to IDNA ASCII form for international domains
              // Skip this for the root zone `.`
              if ($domain != '.') {
                  $domain = convertUnicodeToIDNA($domain);
              }
              // convert the domain lower case and check whether it is valid
              $domain = strtolower($domain);
              $msg = '';
              if (!validDomain($domain, $msg)) {
                  // This is the case when convertUnicodeToIDNA() modified the string
                  if ($input !== $domain && strlen($domain) > 0) {
                      $errormsg = 'Domain '.htmlentities($input).' (converted to "'.htmlentities(utf8_encode($domain)).'") is not a valid domain because '.$msg.'.';
                  } elseif ($input !== $domain) {
                      $errormsg = 'Domain '.htmlentities($input).' is not a valid domain because '.$msg.'.';

As this domain is not a legal IDNA, the conversion fails and the domain is not added.

Did you see a request for r2---sn-5ualdnss.googlevideo.com?

yubiuser · September 16, 2022, 4:39pm

Found it:
https://www.rfc-editor.org/rfc/rfc5891#section-4.2.3.1

The Unicode string MUST NOT contain "--" (two consecutive hyphens) in
the third and fourth character positions and MUST NOT start or end
with a "-" (hyphen).

TheEngiNerd · September 16, 2022, 8:41pm

Hi yubiuser,

Thanks for all the details. Yes, I am seeing many 'r#---sn-" requests from YouTube, about 10-20/hour while viewing videos.

The workaround has been to add a regex expression to filter these sites:

^r[0-9]+---+sn-+[[:alnum:]]+.googlevideo.com$

So far, this seems to be working fine (i.e not blocking videos, just specific ads).

I will look into updating to the development branch, but for right now, I think the regex is doing what I need.

Thanks again for your time!

system · October 7, 2022, 8:42pm

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.