[systemd-devel] PATCH: Integration with Python logging framework

[Zbigniew Jędrzejewski-Szmek]

Hi,

On Mon, Oct 15, 2012 at 11:30:09PM +0300, Marti Raudsepp wrote:
> systemd fields and concluded that the approach I've taken with the
> patch is too simplistic. I'd like to arrive at an API that makes it
> easy to use journal, but is generic enough to be useful for other use
> cases too. These ideas are still fairly raw, but I'm stumped for now.
> 
> ----
> 1. Event data
> One clear use case that's fairly easy to support is supplying custom
> message-specific data, such as MESSAGE_ID. Or details of the event
> that caused the log message; for example, a HTTP request log has
> fields for client's IP address, request URL, User-Agent string, etc.
> The extra= way of providing these works well enough, but there needs
> to be a way to mark which fields are intended to go to systemd and
> which not.
I think that apart from the fields which are already supported
(MESSAGE_ID, CODE_*), there will not be any fields that are widely
used, and that fields will be specific to the message or group of
messages. So I don't see a reason to try to attach the translation
from extra-dict key to systemd field name to the root logger.

[...]
> Python offers LoggerAdapter, but sadly the context they provide is
> only attached to one logger. Since they add fields to the LogRecord,
> the above API already suffices for them.
When I look at LoggerAdapter, it seems to me that it could do the job
just fine. A SystemdLoggerAdapter could add ._systemd_fields dict to
the record object, which would contain the mapping
extra-dict->systemd. It's not particularly efficient, since it
creates an extra LoggerAdapter object per Logger, but logging
is not very fast anyway.

> However, we could make this more powerful and convenient by offering
> context managers and storing the data in a thread-local context. For
> example, web frameworks/servers could add something like this to their
> request dispatcher (obviously simplified):
> 
> def dispatch_request(request):
>     context = {
>         'REQUEST_URI': request.uri,
>         'REQUEST_ADDR': request.ip
>     }
>     with journal.journal_context(context):
>         handler = find_handler(request.uri)
>         handler(request)
> 
> Even better if this were integrated with the logging framework, though
> I'm not certain how it would be implemented. Setting "systemd_fields"
> on the root logger would override any changes made by other modules.
> But this mapping needs transcend beyond just one logger to be
> useful... Something like:
> 
> import logging
> logging.root.systemd_fields = {'ip': 'REQUEST_ADDR', 'uri': 'REQUEST_URI'}
> 
> def dispatch_request(request):
>     context = {
>         'uri': request.uri,
>         'ip': request.ip
>     }
>     with journal.log_context(context):
>         handler = find_handler(request.uri)
>         handler(request)
> 
> ----
> > I've also considered abusing
> > the translation substitution support in logging to put the variable
> > values in into separate fields (in addition to using the composed
> > string as MESSAGE=), since (1) those values are identified by a key
> > for the purpose of substitution and (2) those values are generally the
> > same values you'd want to use for filtering.
> 
> I'm not sure this is a good idea, since it changes even on minor
> changes to the message. In any case, I think it should be evaluated
> independently of Python -- if it seems like a good idea to systemd
> developers, it should be documented and perhaps adopted by other
> packages too, not just Python.
Yeah, this is a no no because it would make the message string
part of the API.

Zbyszek