Hello - I'm new to the pi-hole universe, so I'm a little late to the game.
Id like to piggyback on the OP's request ("option to filter out duplicate domains from the query log section") and break it out into these separate features:
filtering as it currently is implemented (all by domain, all by client) with (Whitelist|Blacklist) button
Checkbox to display the (most recent| n most recent) query by (client|FQDN)
Ability to use RegEX in the search box
Ability to use wildcards in the search box (e.g. *.amazon.com)
Perhaps combine the Long Term Data and Query log menu entries into a single menu node. Query log is essentially a subset of the data which the Long Term Data node provides.
If more robust search functionality were provided (including the above suggestions, and others), there would only be a single location to parse, review, filter, view log data.
To extend this further, if there were a single search location, additional search functionality such as reports, triggers & alerts, and other rich data triggers could be configured to run on a schedule, or once off. A further extension would be the ability to email or message via other messaging apps the events triggered by these logs (i.e. what Papertrail enables users to achieve).
Ultimately, as more and more users learn about pi-hole, its functionality must be extended, become more robust, and easier to use (success has its downsides)!
Finally, thank you for all of your hard work - this is a fantastic product and has reignited my interest in server apps and network administration.
Its functionality is actually pretty robust. The reason for the "normal" and the long-term query log being splited are the (vastly different!) performance requirements. You might have seen that the long-term data needs significantly more time for loading with the data of the last 24 hours is almost delivered instantaneous. We are working on a unification into a single "source of truth" in the new web interface.
I also would love to have a filter option on the query log screen. My stupid "smart" TV keeps sending DNS requests to scribe.logs.roku.com every few seconds and filling up the log so I have a hard time seeing other entries. Yes I know I can add it to the "never show" settings box but that doesn't work for me because I still want to see ALL blocked and ALL allowed queries, just not this one because it is SO CHATTY and hides valid entries when I am trying to remove commercials from TV shows and perform other troubleshooting.
EDIT: now that I think about it more, this feels more like a bug. When you check the "show ALL" boxes in the settings it should actually NOT show anything you entered in the boxes above for exclusion. Otherwise why have those boxes for user entries in the first place? Fixing this would solve my problem, at least, and perhaps OP's as well.
