[systemd-devel] Output from `tee' is not showing up in system journal consistently on some systems

[Mitchel Humpherys]

Hello,

I'm having an issue with logging from some of my user services on some
systems but not others. I asked this on ServerFault a few days ago [1] but
haven't gotten any responses. I apologize if this sort of question is not
allowed here.

On some systems, output from `tee' is not making it into the system journal
consistently. I've reproduced the issue on Manjaro (systemd 249) and Ubuntu
(systemd 245), but I can't reproduce it on Debian 11 (systemd 247). I don't
see any glaring differences between my systemd configurations on Manjaro
and Debian (diff -r /etc/systemd ~/tmp/debian_systemd).

I have full repro instructions over here [2], but I'm pasting the script
and service file here to save you a click:

tee_test.sh

#!/bin/bash

LOGFILE=/tmp/testtee.log
echo > "$LOGFILE"
echo "i like pie" | tee -a "$LOGFILE"

i=2
while :; do
    echo "i shall eat $i slices of pizza" | tee -a "$LOGFILE"
    echo "i shall eat $i slices of pie"
    ((i++))
    sleep 1
done

~/.config/systemd/user/tee_test.service

[Unit]
Description=Test teeing under systemd

[Service]
# same result with either of the following two variants
ExecStart=/usr/local/bin/tee_test.sh
# ExecStart=stdbuf -i0 -o0 -e0 /usr/local/bin/tee_test.sh

# same result with or without this guy
StandardOutput=journal+console

[Install]
WantedBy=default.target


On my Manjaro and Ubuntu systems I'm not seeing all of the "pizza" lines
(logs pasted here [3]). On Debian I see the "pizza" and "pie" lines
interleaved as expected. It also works as expected when I run the script
without systemd on Manjaro.

Any ideas what's going on here? Why am I not seeing all of the output from
`tee' on some systems?

Full version info of the systems I've tested:

Manjaro
systemd 249 (249.4-2-manjaro)
+PAM +AUDIT -SELINUX -APPARMOR -IMA +SMACK +SECCOMP +GCRYPT +GNUTLS
+OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD
+LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT -QRENCODE +BZIP2 +LZ4
+XZ +ZLIB +ZSTD +XKBCOMMON +UTMP -SYSVINIT default-hierarchy=unified

Ubuntu 20.04
systemd 245 (245.4-4ubuntu3.5)
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP
+GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN
+PCRE2 default-hierarchy=hybrid


Debian 11 (WORKS!)
systemd 247 (247.3-6)
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP
+GCRYPT +GNUTLS +ACL +XZ +LZ4 +ZSTD +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2
-IDN +PCRE2 default-hierarchy=unified

Thanks!
Mitch

[1] https://serverfault.com/q/1081172/66928
[2] https://github.com/mgalgs/systemd-tee-output-issue
[3]
https://github.com/mgalgs/systemd-tee-output-issue#logs-in-system-journal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20211026/41e470c9/attachment.htm>