[systemd-devel] Prefix for direct logging

[Arjun D R]

Hi Lennart,

Please help me understand how the journald is figuring out the PID of the
log line. I believe, with the PID, the journald is able to get the
remaining details (process name) from proc fs. I wonder how the journal is
able to get the PID of the log contributor as there can be many processes
spawned under a service! I am trying to understand the journald and it
would be really helpful to know this magic.

Thanks in advance,
Arjun

On Wed, Sep 29, 2021 at 12:14 PM Arjun D R <drarjun95 at gmail.com> wrote:

> Hi Lennart,
>
> That's a good idea but still I would like to have the prefix as it is in
> the journal . I understand it is impossible to bypass the journal and
> expect the direct logging to be the same as journal entries. We can achieve
> it through socket but still we cannot have the luxurious prefix as in
> journal.
>
> I would better like to have a timer unit and corresponding logger unit.
> Based on each service logging capability, let me decide the time.
>
> So it's like:
>
> Timer unit - To trigger the logger service for every n seconds (we can set
> it as per the service logging range).
> Logger Service - Run journalctl -u <service> and the service is designed
> to pass the stdout to the file (using StandardOutput)
>
> By this way, I can avoid the script running for every few seconds which
> runs journalctl for almost 30+ services and redirect it to the respective
> log files one by one at a moment which is impacting the load.
>
> Thanks for the help.
>
> --
> Arjun
>
> On Tue, Sep 28, 2021 at 7:06 PM Lennart Poettering <lennart at poettering.net>
> wrote:
>
>> On Mo, 27.09.21 15:40, Arjun D R (drarjun95 at gmail.com) wrote:
>>
>> > Hi Folks,
>> >
>> > Currently we are using systemd-journald for service logging. We run
>> > journalctl for a bunch of services and redirect those to the custom log
>> > files for every few seconds. This takes up the CPU for that particular
>> > time period since we have lot of IO operations as well. We came to know
>> > that systemd version v236+ supports direct logging
>> > (StandardOutput:file:<log_file>) to the custom log file by the service.
>> I
>> > would like to use that facility but we don't get the prefix that we
>> used to
>> > get when using the journal.
>> >
>> > Is there a way to prepare a custom patch locally to add the necessary
>> > prefix to the stdout before writing to the custom log file? Is that a
>> good
>> > idea? Any other suggestions?
>>
>>
>> You might define a socket unit 'prefixlogger at .socket' like this:
>>
>>   [Unit]
>>   StopWhenUnneeded=yes
>>
>>   [Socket]
>>   ListenFIFO=/run/prefixlogger.fifo.%i
>>   Service=prefixlogger@%i.service
>>
>> And then a matching service 'prefixlogger at .service':
>>
>>   [Service]
>>   StandardInput=socket
>>   StandardOutput=file:/var/log/foo.log.%i
>>   ExecStart=sed -e 's/^/foo:/'
>>
>> And then in the services that shall run with this:
>>
>>   [Unit]
>>   Wants=prefixlogger@%N.socket
>>   After=prefixlogger@%N.socket
>>
>>   [Service]
>>   ExecStart=/my/service/binary
>>   StandardOutput=file:/run/prefixlogger/fifo.%N
>>
>> (This is all untested, might need some minor changes to actually work,
>> but you get the idea).
>>
>> So what this does is this: first we define a little socket service
>> that can be initialized easily a bunch of times: it listens on a FIFO
>> in the fs and everything it reads from it it writes to some log
>> file. The service is just an invocation of "sed" with standard input
>> being the fifo and standard output being the log file to write to.
>>
>> You then use it by using StandrdOutput=… in your main unit, to connect
>> its stdout/stderr to that fifo. Also, you add deps so that each time a
>> service that tneeds this starts the log prefix service socket for it
>> starts too.
>>
>> Lennart
>>
>> --
>> Lennart Poettering, Berlin
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20210929/8c68eda3/attachment.htm>