Request: show regex, responsible for blocking

@DL6ER: You have created a very useful tool to help identifying blocked domains, as a result of CNAME blocking. Hope to see it soon in Beta5.

I would like to make a similar request for regex blocking. Currently the web interface shows:


It would be very helpful to show the regex entry, responsible for blocking the query.

As discussed here, no final decision made, you could use the tooltip entry to show the additional information, this to avoid providing to much information on the screen, making analysis harder.

You can enable regex debug mode and this information will be logged in /var/log/pihole-FTL.log

I know that, which means the information is available to pihole-FTL. @DL6ER made some changes to show the CNAME in the web interface, causing a block. The same method could be used to show the regex, causing the block, thus eliminating the need to enable debug mode

Using debug mode, the output is a single line in a log, retained for a few days. Are you proposing to add this information to the long term database (which is where the dashboard information is obtained on an FTL restart)?

In effect, making debug mode the default and storing the results in the long term database?

I have no idea how @DL6ER has achieved the CNAME solution, the result however is the CNAME is visible in the web interface (see screenshot in first entry of this post. I assume the same method can be used to display the regex. If this implies a database change, so be it, but you have to ask @DL6ER what he has changed to make this great improvement.

No modification to the database as we’re not storing this extra information at all. The regex filters exist only in compiled form in FTL. There is no copy of the string available that was used to create the compiled regex.

In debugging mode, we reserve some extra space that holds all the strings. Instead of in debugging mode, we could store the strings always. However, I have some more issues with this as domains are typically small, however, we’ve seen a lot of regex filters lately that caused explosions of the group management pages. Filters consisting of 200 (or even more!) characters would clearly not fit here.

I can maybe see why this is useful information and maybe we add it somehow…
However, I can already see the next request incoming: Show which blacklist record or blocking list caused a blacklisted/gravity domain to get blocked… For these two, we cannot easily add code that determines which entry was causing the blocking this information is not available in the highly efficient tree lookup method we’re currently using. I don’t want to reduce performance for this extra information.

All this means: Maybe. I don’t know yet or how to do it best.

As opposed to the CNAME entry, where it is possible to show the domain, I understand showing the regex may be impossible, due to length.

Would it be possible to create clickable link, opening a new window to the regex blacklist page, highlighting the correct regex?

Just an attempt to make debugging of blocked entries a lot easier for the user.

Yes, this is what I’m currently investigating.

May I suggest you open the link in an iframe (not a new window or a new tab)?

Iframes are forbidden inside the current web interface. Also, the table doesn’t like having iframes in it.

Queries blocked by a regex filter (either during CNAME inspection or directly) will present a link. Once you click on this link, it will send you to the domain group management page highlighting the matching regex filter.

This is part of new/CNAME_inspection_details which should get reviewed and merged soon.

I carefully implemented this without any performance hit. However, this is only possible for regex filters as we need to walk them all until we find one that matches. This is not possible for the exact blacklist or gravity.

@DL6ER I’ve upgrades to the latest version and also seeing the link to “Jump to regex” now. It’s a bit confusing that the link appears on all entries regardless of the status (see screenshot from @jpgpi250).

Can you adjust the code so that the link just exists when the status contains something with ‘regex’ in it?

I had some serious problems with the system where we tested the CNAME_inspection_details, so I reimaged (etcher) and tested again. It all appears to work as you described, so I deleted my earlier post, describing the problems, since they apply only to that system (already overwritten).

I agree, this is confusing. it would be an improvement to only show the link when it will work. You might even consider to simply turn the word regex into a link, no regex, no link, no need to add Jump to regex… (If possible).

Will this work for regex whitelist?

As you may have guessed, this is not intended :wink: Something went wrong and this just teaches me again to not “quickly push something before heading to work”. I will push an update to FTL now.

This also looks kinda strange:

I don’t think I like it. Hint: I have not pushed this change upstream.

No, there is no way to identify if something was whitelisted on the dashboard. Whitelisted queries are just “not blocked”.

pihole -up:

  [✓] Deleting existing list cache
  [✗] DNS service is not running
  [✗] Job for pihole-FTL.service failed because the control process exited with                                                      error code.
See "systemctl status pihole-FTL.service" and "journalctl -xe" for details.
  [✗] DNS resolution is not available

  Unable to complete update, please contact Pi-hole Support

there was a retry counter …

sudo service pihole-FTL status

● pihole-FTL.service - LSB: pihole-FTL daemon
   Loaded: loaded (/etc/init.d/pihole-FTL; generated)
   Active: failed (Result: exit-code) since Tue 2020-02-04 19:29:39 CET; 1min 56s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 7120 ExecStart=/etc/init.d/pihole-FTL start (code=exited, status=1/FAILURE)

Feb 04 19:29:34 raspberrypi systemd[1]: Starting LSB: pihole-FTL daemon...
Feb 04 19:29:39 raspberrypi pihole-FTL[7120]: .....
Feb 04 19:29:39 raspberrypi pihole-FTL[7120]: Not stopped; may still be shutting down or shutdown may have failed, killing no
Feb 04 19:29:39 raspberrypi systemd[1]: pihole-FTL.service: Control process exited, code=exited, status=1/FAILURE
Feb 04 19:29:39 raspberrypi systemd[1]: pihole-FTL.service: Failed with result 'exit-code'.
Feb 04 19:29:39 raspberrypi systemd[1]: Failed to start LSB: pihole-FTL daemon.

sudo service pihole-FTL stop && sudo service pihole-FTL start -> OK

result (you are a genius):
query log:


after clicking the link:

Whats about turning the whole status ‘Blocked (regex blacklist)’ into a link? I think this would be optimal.

Well, maybe not optimal but we obviously need some obstacle to signal users that this is clickable:

Adding a small ‘This is a link’-Icon?

How so? Like this?

Yapp.