First of all, The request is to format the messages in
pihole-FTL.log only, not pihole.log. The pihole.log is almost constantly updated, rsyslog triggers with actions on such a busy file don't work very well, rsyslog actions (omprog) have trouble to keep up with the rate the log entries are added.
When using rsyslogs imfile module, all messages, registered in the log that is monitored, are replicated to both syslog and messages, unless you have a proper filter to stop processing.
example, an unbound message, logged in a logfile (unbound.conf with the setting logfile: /log/unbound.log), thus not controlled by syslog, but rsyslog monitored using imfile looks like this:
May 05 19:13:57 unbound[4456:2] info: resolving a28-67.akam.net. AAAA IN
May 05 19:13:57 unbound[4456:2] info: response for adns1.akam.net. AAAA IN
May 05 05:32:11 unbound[6918:0] info: error sending query to auth server 2001:501:b1f9::30 port 53
unbound clearly tags all messages as info, thus in order to stop further processing, avoiding any actions, despite the word 'error' in the last message, the conf file starts with:
if $msg contains 'unbound[' and ($msg contains 'info:' or $msg contains 'notice:' ) then stop
Why like this? The unbound log, configured in unbound.conf (logfile: /log/unbound.log), still contains all messages, even the info messages. This is useful when trying to identify an unbound problem. The rsyslog imfile configuration for the unbound log than stops processing the message, if 'info' or 'notice' is detected, preventing the message to be entered in the syslog and messages logs. Since there is no further processing, action triggers are never executed.
I would suggest to use the unbound format, since a lot of pihole users also run unbound.
Be aware that not all versions of rsyslog (depending on the distro) have the case sensitive option, e.g. use
contains_i, to ignore case doesn't always work. Unbound uses all lower case. This document also lists al the severities, you mentioned, in lower case.
The different process names are not a problem, the rsyslog conf file simply needs to provide stop filters for all the processes the user doesn't want to be notified (rsyslog action) about.
Your comments are correct, however:
- currently the pihole-FTL log cannot be used efficiently (with rsyslog - imfile), because of the lack of any filter options, resulting in the fact that all log entries from pihole-FTL.log, are duplicated in both syslog.log and messages.log
- looking at my pihole-FTL log today, after 19 hours (logrotation at midnight) the log contains only 122 lines.
- 14 of these messages already contain
Notice (should be
notice - lower case).
- 11 of these messages contain
Resizing, this is clearly an
- 11 messages are the result of
sudo service pihole-FTL stop, clearly
- 68 messages are the result of
sudo service pihole-FTL start, clearly
- 10 messages containing
- 4 messages containing
Very few messages left to write a filter for...
This doesn't mean the request is useless (very few messages under normal conditions). I want to catch the messages that will appear, when something is wrong, for example the incorrect regex warning (or other messages that wouldn't appear normally, for example)
This implies of course that all messages, generated as a result of activating a debug option in pihole-FTL.conf should get
debug, this to be able to create an effective stop filter. If a debug option is enabled, the user will be looking at the pihole-FTL log, in search for a problem, probably instructed to do so by a developer.
under normal conditions, I never see any sqlite3 messages in the log. I thus wouldn't add a stop message for sqlite3 messages, because something might be wrong, worth analyzing.
I don't expect the code for dnsmasq and sqlite3 to be modified, I only request the messages you specifically added to pihole-FTL to be modified...