Improve domain list inout fields

Confusion for the users arises when they can enter either domains or regex into the same box.

By having one entry box for domains, and a separate entry box for regular expressions, I think the intent and function of the page is greatly improved. We can put the input notes into the box in grey text as we do now.

2 Likes

I'm actually beginning to wonder about things myself, now.

When entering a domain and selecting the "wildcard" type, it is converted to a regex.

So, entering test.com (or, indeed *.test.com) and choosing wildcard, the entry is converted to (\.|^)*\.test\.com$

Entering *.test.com and choosing regex parses stores it as a regex value of *.test.com, which has a different meaning to (\.|^)*\.test\.com$

Really, wildcard is just a helper function for someone to be able to enter the regex for a wildcard block, without actually knowing how to regex.

1 Like

I think I have to agree with @jfb about separating the inputs out to two distinct boxes. One for exact/wildcard, and another for regex (marked as "advanced" or something)

3 Likes

In the entry choices for the domain section, we could put entry choices labeled differently:

exact blocking *this domain only"

wildcard blocking "this domain and all subdomains"

1 Like

Agreed. OK, we will work on this... though, as ever with this kind of things, no guarantees on timescale.

Soon™ is as good as you'll get from me.

Of course, if anyone here reading this has (even the smallest amount) of skill in PHP/JS/HTML, then please feel free to submit a Pull request over at the repo. We gladly welcome contributions from the community, and we don't bite all that hard...

2 Likes
1 Like

I think this would be very helpful.

I'm late to the party, but better being late than not showing up at all.

  1. We had validation for domains (and wildcard-style domains!), but we lost this during the transition from the former to the new domain pages. Definitely something to re-implement.

  2. Regex is and was never validated in the dashboard.
    This is very complicated to do in JS or PHP (at least for the Extended Regex dialect), just checking whatever PHP or JS may offer like PCRE or ECMAScript won't help because any dialect has its specific specialties.
    We are working on somehow showing if FTL detected an issue (alongside some helpful non-crypting and actually human-readable description of what was wrong!), however, this is a bigger thing and will likely be part of Pi-hole v5.1. We have had regex for a while now and I see no immediate reason why this feature would need to be added now as this might push back the official v5.0 release by weeks, otherwise.

  3. Wildcards are regular expression filters. They cannot easily be distinguished. I'm okay with splitting the currently one input-field interface into a two input-field one, but I will then also remove the dropdown menus at all and replace them by a checkbox (wildcard).

  4. This should be stressed:

    I have seen the possible confusion and only re-added it one week ago as someone complained it was missing. Now it is back and we're complaining about it being confusing. What should I do? Yes, it is really only a helper function. Wildcard domains - as such - do not exist. This helper is just preparing a suitable regex for you that will achieve the same thing.

2 Likes

I think having two input fields (domain and regex) and a checkbox next to the domain-field (labeled something like 'convert to wildcard domain') in conjunction with a explanation box would reduce the confusion.

Coming over to this after just setting up a wildcard whitelist entry (that I had done wrong apparently) I think splitting it into two boxes and offering a short description/explanation as to what a domain vs wildcard entry is would go a long way to clearing up the confusion.

I know personally most places that I have entered a wildcard anything that is not regex I need to include some sort of wildcard character (like *, or ?). This is mostly from dealing with cmd and PoSH, but I think it is fair to assume that many (most?) users will not be super comfortable with regex, may not know it at all, and are probably used to including something in the wildcard spot. As such would it be terribly difficult to implement a wildcard character (I nominate '*'), and when it is found in an entry assume the user wants a wildcard entry and convert it? Then you can get rid of the option altogether. Essentially instead of offering the option to the user, use the inclusion of something like * (which is otherwise an illegal character in a domain) as the toggle for wildcard or not.

As such you would end up with a regex entry box, and a domain entry box that could take either "example.com" for a domain match, or "*.example.com" for a wildcard match. The default text in each could allude to what to do:

Domain: Add a domain (example.com, sub.example.com, or *.example.com)

Regex: Add a regular expression

Additionally, I think a simple link to the documentation for more information would then suffice instead of including it on the page.

What did you enter, what was the response and what happened with blocking? What tells you that you had done it wrong?

I had entered *.example.com, which allowed the page I was trying to get to to resolve. So I guess not wrong in that it did not work, but wrong in that the regex was formed with the * in it which I learned after seeing some of the issues posted here around running pihole -q or -d. As a note I did not have these issues, but also had not run those commands after making the change.
As someone who rarely if ever uses regex I would have had no idea it was formed wrong.

There was a fix pushed today to address this specific case. That is why I asked about something being done "wrong". Can you give more detail on what you entered and what site you wanted to block? Also what sites made it through after you tried the block?

I was actually whitelisting myuhc.com. I entered *.myuhc.com (thinking of it as a wildcard, emails from them often have all sorts of subdomains and such, in this case it was "click.e-notifications.myuhc.com") which did indeed work to whitelist the site. So it was not broken and did fix the previously blocked domain. I did update my install afterwards, so it worked on the version I had installed from about a week and a half ago, and continues to work on the current version. That said I did not run pihole -q or pihole -d after adding it.

If it works as you expected or intended then it's working as it's supposed to.

There's really isn't a "correct" way to do it. If it works, then that's right for you. There aren't any issues with awk locking and crashing because awk isn't there anymore. If you're on the release/v5.0 branch and have pihole -up in the last few hours then you have the fix. You can run pihole -q or -d and it will all work.

In fact, please run pihole -d and let it run. If you want to post a token that is shown at the end of the process then we can take a look as well.

I ran pihole -d and it ran quickly with no issues. Uploaded the log with token https://tricorder.pi-hole.net/38xh3sn7m6

Glad to hear that it is working as it is supposed to. Just trying to add an idea to the topic of how best to present the regex/wildcard UI to users. As it stands now I don't really need to interact with it too much. My pi-hole install is pretty set and forget except for when I need to unblock a domain here and there. Though with the new group settings in v5.0 I will probably setup a few different groups for more granular control. Overall it works really well, especially since the gravity database update performance was patched, it is much faster with no downtime at all when it does updates.

Thank you for the debug log. It is exactly what I was looking for. A good number of different types of lists and overrides.

Glad everything is working for you.

This is true, can you give a little more detail about what that statement means to the topic at hand?

It's also a way to have five aces in a poker hand, but that doesn't help a DNS server much.

1 Like

As regards regex, yes. But * is widely regards as a wildcard char outside of regex.

At the end of the day, what we have is two separate definitions of wildcarding, which are currently enterable into the same box.

I would posit that the majority of users see * as a wildcard, rather than .

But we can change things, and I think we came up with a sensible solution in this thread. It just needs to be implemented.

Agree, and splitting the entry in the web interface is good. But for now it works as intended and no longer causes regex errors or crashes of FTL.