[systemd-bugs] [Bug 77013] RFE: journald to send logs via network

[bugzilla-daemon at freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=77013

--- Comment #4 from Duncan Innes <duncan at innes.net> ---
Thanks for keeping this open.  I was confused about what part did the sending
and what did the receive.

As for the output formats and extra tags - here goes.

The use case for JSON formatting is to send logs to alternative aggregators
(such as Logstash as mentioned in comment #3).  The ability to receive logs in
separated format rather than log lines makes it much easier for these systems
to parse entries and stick them in whatever database is being used.

The use case for extra tags I would say is similar to Puppet/Foreman hostgroups
or classes.  Systems know quite a lot about themselves which the log aggregator
is going to have a hard time figuring out.

Client systems know if they are dev, test, uat or production.
Client systems know if they are in the DMZ (potentially)
Database servers know that they are database servers
Web servers know that they are web servers
and so on . . .

If each client can add some tags that provide context to the log entries,
searches through logs can be made very much more useful.

I could search for all IPTABLES denials on my web servers.
I could search for all failed login attempts on my DMZ servers.

Strictly speaking, the log comes from a single machine, but being able to group
these machines arbitrarily (as happens naturally on a large estate) will allow
an extremely powerful context search on the log database.

Why not get the aggregator/parser/indexer to add these fields?  These machines
will not necessarily know all the details that the client might want to add. 
The client already knows these details, or can have them set via whatever
config management tool is being used.

Overall system loads will also be reduced by clients having a config entry that
(for example) hard codes "cluster": "WebApp3" to be added to the log entries
rather than having the aggregator performing repeated calculations or lookups
on whatever LDAP, node classifier or other method is used.

I don't mean to unduly extend the features of log shipping, but allowing a
couple of output formats and some extra fields to be pushed would be a big
benefit to large scale system users.  Especially when the first point of
inspection of aggregated logs is potentially a script/automated process rather
than a SysAdmin.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/systemd-bugs/attachments/20140403/f88c3669/attachment-0001.html>