[systemd-devel] [PATCH] core: switch to journal when socket is listening

[Umut Tezduyar]

On Thu, Jun 13, 2013 at 9:26 PM, Umut Tezduyar <umut at tezduyar.com> wrote:
> systemd starts using journal as soon as the journal
> socket is in listening state instead of waiting for
> journal's socket to switch to 'running' state.
> ---
>  src/core/manager.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/src/core/manager.c b/src/core/manager.c
> index f16621a..cebc43e 100644
> --- a/src/core/manager.c
> +++ b/src/core/manager.c
> @@ -2613,7 +2613,8 @@ void manager_recheck_journal(Manager *m) {
>                  return;
>
>          u = manager_get_unit(m, SPECIAL_JOURNALD_SOCKET);
> -        if (u && SOCKET(u)->state != SOCKET_RUNNING) {
> +        if (u && SOCKET(u)->state != SOCKET_RUNNING &&
> +                 SOCKET(u)->state != SOCKET_LISTENING) {
>                  log_close_journal();
>                  return;
>          }
> --
> 1.8.3
>

Hi,

Due to a race condition (at least I think) between systemd and
journald, systemd might keep logging to kmsg even though journal is up
and running. Systemd switches to using journal for logging when both
journal's socket and service are in running state. Occasionally  I
have ended up finding my system in the following state after booting
to basic.target.

systemd-journald.service   loaded active   running     Journal Service
systemd-journald.socket    loaded active   listening   Journal Socket

At this point journal is up, program's stdout/err is sent to journal
(can be seen by journalctl) but systemd still thinks journal is not
usable for its own logging and keeps logging to kmsg.

If you are using kernel < 3.5, the effect of the problem is not being
able to see systemds logs in journalctl.

-------------------------------

I think what is happening is both journald and systemd are racing for
the EPOLLIN event on the journal's sockets. If journal is receiving
EPOLLIN events all the time, then systemd will never get notified of
the activity on the sockets and never switch the socket to running
state.

If I disable the EPOLLIN event registration on the journald side, I
can see that systemd-journald.socket switches to running state %100.

Since systemd-journal.socket and systemd-journal.service are starting
on boot up anyways, I don't see the necessity of waiting for
systemd-journald.socket to go to running state.

If anyone would like to get more information, I can reproduce the
problem pretty often. I have tried both kernel 3.4 and 3.8 as well as
different ISAs.

Thanks