[systemd-devel] Linux Journal API/client lib

[Kay Sievers]

On Fri, Dec 2, 2011 at 16:14, Rainer Gerhards <rgerhards at gmail.com> wrote:
> On Fri, Dec 2, 2011 at 2:49 PM, Kay Sievers <kay.sievers at vrfy.org> wrote:
>> On Fri, Dec 2, 2011 at 13:59, Rainer Gerhards <rgerhards at gmail.com> wrote:
>>> as you probably know, I am not a big fan of the journald proposal, but
>>> that's not the point of my question. I am thinking about how to
>>> integrate journal data into a syslog logging solution.
>>
>> You know that the syslog daemon will still see exactly the same log
>> messages from all clients as it did before, right? The /dev/log file
>> descriptor that systemd passed to the syslog daemon at startup, will
>> still carry all the same things regardless of journald's actions.
>
> Does that mean /dev/log will also receive messages submitted via the
> *new* API you define? If so, is the format documented somewhere (or
> intended to be)?

/dev/log will be read by journald. The syslog.socket filedescriptor
that the syslog daemon receives, will be provided by journald and have
all the messages which are received by /dev/log and the human readable
part that is received over the native journal interface.

The /dev/log the syslog daemon sees, will not be the same /dev/log
which glibc talks to.

>>> So I have a
>>> couple of questions in regard to accessing the journal. The most
>>> important one is how to read information from it. From what I have
>>> read, a client API is planned. I know git has working code, but this
>>> code is tied into systemd and is not a lib.
>>>
>>> So I wonder how a non-systemd app would gain access to journal data.
>>> Do you actually plan a client library (-ljournal) ?
>>
>> Yes, a shared library will be provided.
> It's probably a bit early, but: do you have some idea when this will
> be released (maybe around F17)?

That's the plan.

>>> Or anything else? Also, in order to read the complete
>>> journal, is it necessary to run under the root user or can lower
>>> privileges be used (and, if so, which)?
>>
>> All entries are written to individual journals, a separate one for the
>> system and one for every login uid (user) that submitted entries.
>> Every login uid can read their own journal, but not directly write it.
>
> But there will be one journal that a root admin can pull for all log
> entries? Or does this mean that, in order to obtain all entries, the
> system journal file plus all journal files for all users must be read?
> If so, is it intended that the API/lib handles that?

The library already merges all files the caller has access (unix file
permissions) to. It's transparent to the reader, if it can read them,
they will be included in the stream.

>>> These questions were mostly related to the syslogd pulling data from
>>> the journal to transmit it some remote site (or do some other
>>> transformation). Now let's assume the contrary, syslogd receives data
>>> via the network and wants to store it inside the journal. Will there
>>> be an API provided for that case as well?
>>
>> Yes, sure, it can just forward things to the journal. Along with what
>> they log, they will just have some metadata of the forwarder added.
>
> So now let's assume I have pulled some log messages from system A, and
> transport these via syslog to system B. Now I want to consolidate the
> log on system B. So what I need is to have an exact duplicate of what
> is present on A also present on B (especially the metadata). That
> means I can write into B's journal exactly what was on A, including
> the *trusted fields*? (or let's for a moment assume that A  does not
> run journald, but I know A's hostname via RFC5425 X.509-based auth and
> so this info is known to be correct - many scenarios along these
> lines).

We haven't thought about any of the details how to handle possible
trusted log merge. It's surely possible to do something like that, but
we have no specific ideas as of now.

In the end it's again a bit like git, and the model journal can do
across the network is like 'git pull' from other hosts. So there is
certainly the possibility of having a 'syslog journal gateway' that
provides the syslog 'commits' which are to be merged.

Kay