[systemd-devel] High CPU usage of journald

Holger Hans Peter Freyther holger at freyther.de
Mon Feb 18 10:52:39 PST 2013

On Mon, Feb 18, 2013 at 08:47:44AM -0800, David Strauss wrote:

Dear David, all,

> It's possible that a lot of it could also allocate on the stack or use
> stack-style management of a pool in heap. This is pretty
> performance-critical code, and we're seeing similar CPU overhead.

I think upstream doesn't care about the CPU overhead of the journald. I
had a quick look on my notebook with the same perf command and _int_malloc
is noticable in the trace as well.

In OpenBSC/Osmocom we have our struct msgb (a clone of struct skb) and
we can pull and push from the allocated memory. E.g. to replace a LayerX
header or prepend another header.

I browsed through journald-server.c:dispatch_message_real and there are
several things one could do... there are certainly a lot more items.

1.) Several things are done over and over again... e.g. calls to the
have_effective_cap.. it is unlikely that it will change after the journald
has started.

2.) Avoid allocation part one...

                r = get_process_cmdline(ucred->pid, 0, false, &t);
                if (r >= 0) {
                        cmdline = strappend("_CMDLINE=", t);

                        if (cmdline)
                                IOVEC_SET_STRING(iovec[n++], cmdline);

So instead of the strappend (which will do strdup..) one could make sure
that 't' has enough space to prepend the _CMDLINE= or the _EXE, or the
other strings.

3.) Looking at code like... get_process_cmdline

  asprintf(&p, "/proc/%lu/cmdline", (unsigned long) pid)

So first of all /proc/%lu/ is created over and over again for exe, comm,
cmdline, sessionid, cgroup... With a different API one could easily avoid

4.) Still at the same method..

        if (max_length == 0) {
                size_t len = 1;
                while ((c = getc(f)) != EOF) {
                        k = realloc(r, len+1);
                        if (k == NULL) {

is most likely not the most effective way to handle the allocation. Or
maybe call get_process_cmdline with a max length?

5.) I wonder if some of the information could be sent from the systemd
to avoid the work in the journald...

I will end up commenting out most of dispatch_message_real and check if
the CPU down drops to a low single digit. I would assume it will.


More information about the systemd-devel mailing list