[systemd-devel] PATCH: Integration with Python logging framework

[Marti Raudsepp]

On Tue, Nov 13, 2012 at 12:23 PM, Zbigniew Jędrzejewski-Szmek
<zbyszek at in.waw.pl> wrote:
> I went ahead and merged the patch with some small changes, please have
> a look.
> JournalHandler is already useful right now

Cool, thanks! Agreed, the rest could just be built on top of this code.

On Tue, Nov 13, 2012 at 12:43 PM, Zbigniew Jędrzejewski-Szmek
<zbyszek at in.waw.pl> wrote:
> On Mon, Oct 15, 2012 at 11:30:09PM +0300, Marti Raudsepp wrote:
>> 1. Event data
>> One clear use case that's fairly easy to support is supplying custom
>> message-specific data, such as MESSAGE_ID. Or details of the event
>> that caused the log message; for example, a HTTP request log has
>> fields for client's IP address, request URL, User-Agent string, etc.
>> The extra= way of providing these works well enough, but there needs
>> to be a way to mark which fields are intended to go to systemd and
>> which not.

> I think that apart from the fields which are already supported
> (MESSAGE_ID, CODE_*), there will not be any fields that are widely
> used, and that fields will be specific to the message or group of
> messages. So I don't see a reason to try to attach the translation
> from extra-dict key to systemd field name to the root logger.

Yeah, the intention was that the app developer wouldn't attach these
to the root logger, but to the specific logger where they are using
those extra fields.

>> 2. Context data

>> Python offers LoggerAdapter, but sadly the context they provide is
>> only attached to one logger. Since they add fields to the LogRecord,
>> the above API already suffices for them.

> When I look at LoggerAdapter, it seems to me that it could do the job
> just fine. A SystemdLoggerAdapter could add ._systemd_fields dict to
> the record object, which would contain the mapping
> extra-dict->systemd. It's not particularly efficient, since it
> creates an extra LoggerAdapter object per Logger, but logging
> is not very fast anyway.

I think you missed my point: it should be possible for the caller to
specify context fields and *values* without having to plaster all
callee log messages with an extra={} dict, nor requiring them to wrap
every logger they use in a SystemdLoggerAdapter.

While developing some Django applications, I found logging to be
almost useless because many layers of the application don't (and
shouldn't) have access to details of the request. E.g. errors from a
third party PDF renderer module. Yet request context details are
crucial when analyzing logs: did two subsequent log messages come from
the same request or two completely unrelated requests? Which user
caused this message to be logged?

There's one place that's well positioned to add these fields: the
request dispatcher. It should be possible to add these fields to all
log messages by any callee by only patching one place. If we can make
it as easy as that, I can see common upstreams like web frameworks
coming with nice journal integration by default. If they have to
change every module to supply some journal fields -- forget it.

Regards,
Marti