[systemd-devel] Journald logging handler for Python 3 and AsyncIO integration

[Ludovic Gasc]

2015-05-02 17:22 GMT+02:00 Jörg Thalheim <joerg.systemd at higgsboson.tk>:

> On Sat, 2 May 2015 16:31:44 +0200
> Ludovic Gasc <gmludo at gmail.com> wrote:
>
> > 2015-05-02 16:18 GMT+02:00 Zbigniew Jędrzejewski-Szmek
> > <zbyszek at in.waw.pl>:
> >
> > > On Sat, May 02, 2015 at 03:34:52PM +0200, Ludovic Gasc wrote:
> > > > Ok, my bad, I didn't see JournalHandler class to use with Python
> > > > logging:
> > > >
> > >
> http://www.freedesktop.org/software/systemd/python-systemd/journal.html#journalhandler-class
> > > >
> > > > Nevertheless, my question about communication between Python and
> > > > journald remains.
> > > Can you rephrase the question? I don't quite understand what
> > > functionality you're missing from
> > >
> > >
> http://www.freedesktop.org/software/systemd/python-systemd/journal.html#systemd.journal.send
> > > .
> > >
> > >
> > In AsyncIO, when you interact with the outside world, you need to use
> > "yield from", for example:
> >
> https://docs.python.org/3/library/asyncio-stream.html#tcp-echo-client-using-streams
> >
> > "yield from" means to the Python interpreter to "pause" the coroutine
> > to work on another coroutine until data arrive.
> > To be efficient, I/O should handled by AsyncIO, even if some
> > workarounds are possible if the raw socket isn't accessible.
> >
> > If this pattern is useful for HTTP requests, for logging where you
> > shouldn't wait a return, it isn't very critical, especially if you use
> > syslog protocol with UNIX socket.
> >
> > Nevertheless, to be sure, I wish to understand how the communication
> > works between my process and journald daemon.
>
> AsyncIO was not their, when these bindings were written. Inventing a
> asynchron API on top of a synchron API (as provided by sd_journal_sendv)
> might be hard.
> So I have two ideas here:
>
> 1. use a background thread, which writes messages to journald from a ring
> buffer and fill this ringbuffer from your application.
>

As mentioned by Zbigniew, it should be slower that send directly,
especially because with Python we have GIL for threads.


> 2. write to the unix socket /run/systemd/journal/socket directly with
> python and asyncIO,
>    this is not as hard as it sounds because it is still some kind of text
> protocol. The coreos guys did this in go here:
>    https://github.com/coreos/go-systemd/blob/master/journal/send.go#L68


It's often the way followed by the AsyncIO community, rewrite or cherry
pick a protocol business logic and handle sockets with AsyncIO directly.
Nevertheless, in the case of UNIX socket, especially with a datagram packet
where you don't wait to send something, you have almost no benefit to adapt
that for AsyncIO.

Thank you guys for theses answers, I'm continuing to play with that, I
think you did everything I need to work.
If I have an issue, I'll open a pull request and post the patch on this
mailing-list.


>
>
> _______________________________________________
> systemd-devel mailing list
> systemd-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/systemd-devel/attachments/20150502/eb38f3a7/attachment-0001.html>