[systemd-devel] RFC: filter and search journalctl

[Zbigniew Jędrzejewski-Szmek]

On Fri, Aug 07, 2015 at 11:53:13AM +0200, Sebastian Schindler wrote:
> Grep-ing seems to be the only solution to find log entries if you don't fully
> know what you're looking for. For example: You want to see all entries
> containing a certain MESSAGE that gets enriched with additional information
> during the logging process:
> 
> MESSAGE=host <HOST> has closed connection <CONNECTION_ID>
This is a bit contentious, but at least I would like to see some
grep functionality implemented directly in journalctl.

> At the moment you have no option to look for this kind of information unless
> someone has set something like  MESSAGE_ID you can filter for. There are several
> use cases using this pattern of thinking:
> 
> * there's no option to show all set FIELD keys in the current journal, although
>   this information is encoded in the header of each journal file
This should be easy enough to add.

> * there's no support for negated filtering, you can't easily hide output of a
>   certain unit which is creating too much noise
This has been on the todo list for a long time.

> * there's no support for regular expressions (except for the --unit option),
>   this is especially problematic when you're looking for certain MESSAGEs
> * there's no option to show all entries containing a certain field
> * logical expressions are somewhat hard to read/write because parenthesis can't
>   be used to enforce certain logical expressions
journalctl is supposed to be simple. Arbitrarily complex queries
are not something that is ever going to be well supported. Like
David said, there's ELK and other stacks for that.

> What do you think about a query language for journalctl that allows more
> powerful search options? This could be introduced without ignoring the
> capabilities the journal file format has to offer. Are there maybe already plans
> to introduce something alike into journalctl? Do some people here have
> experience with query languages for such a use case? Things come to mind like
> PCAP filter, SPARQL, Lucene or the SPHINX Query Language.
It really depends. I think that anything which directly queries
information from the journal headers yes, regexps should be discussed,
anything more complicated no.

Zbyszek