[systemd-devel] [EXT] Journalctl exits under heavy logs

[Ravindran Shanmugam]

Thanks, but how does that relate to these failed messages?

On Mon, Mar 22, 2021 at 1:47 PM Ulrich Windl <
Ulrich.Windl at rz.uni-regensburg.de> wrote:

> >>> Ravindran Shanmugam <ravindran.shan at gmail.com> schrieb am 20.03.2021
> um
> 20:50
> in Nachricht
> <CAC4QDGdbDvhnJ3E+mCDr+s3GZQ+eztLbD=fQUKGOUeeQy=Y_ow at mail.gmail.com>:
> > Hi,
> >
> >
> >
> > Bug / Issue:
> >
> > Under heavy logs entering the systemd-journald, journalctl Exits with one
> > of the following error messages:
> >
> >
> >
> > Failed to iterate through journal: Bad message
> >
> > Failed to get realtime timestamp: Bad message
> >
> > Failed to get monotonic timestamp: Bad message
>
> In theory with recent machines, you can get two identical timestamps even
> with
> nanosecond resolution on subsequent calls.
> I learned that some months ago.
>
> >
> >
> >
> > The systemd is at version 243
> >
> >
> >
> > Is this an Upstream bug which was fixed.?
> >
> >
> >
> > N.B:-
> >
> > As of now, not planning to upgrade to latest systemd version(247) for
> some
> > legacy reasons.
> >
> >
> >
> >
> >
> > Repro steps:-
> >
> > 1)Change the following 2 options in the file "/etc/systemd/journald.conf"
> > from
> >
> >
> >
> > RateLimitIntervalSec=1s
> >
> > RateLimitBurst=2000
> >
> >   to
> >
> > RateLimitIntervalSec=0
> >
> > RateLimitBurst=0
> >
> >
> >
> > to turn off rate limiting, and then kill the "systemd-journald" process
> so
> > that the new one
> >
> > will be spawned and picks up the new jounald.conf
> >
> >
> >
> >
> >
> >
> >
> > 2) Start ‘journalctl’ with this command:
> >
> >   "/bin/journalctl -o json
> >
>
> --output-fields=_SOURCE_REALTIME_TIMESTAMP,__REALTIME_TIMESTAMP,SYSLOG_IDENT
> > IFIER,_SYSTEMD_UNIT,SYSLOG_FACILITY,MESSAGE,PRIORITY
> > -f --no-tail > /dev/null 2>&1 &"
> >
> >
> >
> >
> >
> >
> >
> > 3)Create following bash script and run it as "test.sh 100 1"
> >
> > --------------------
> >
> > #! /bin/bash
> >
> >
> >
> > if [[ ! -e /tmp/lines1.txt ]]; then
> >
> >     for i in $(seq 1 50001); do echo "abc1-$i" >> /tmp/lines1.txt; done
> >
> >     for i in $(seq 1 50001); do echo "abc2-$i" >> /tmp/lines2.txt; done
> >
> >     for i in $(seq 1 50001); do echo "abc3-$i" >> /tmp/lines3.txt; done
> >
> >     for i in $(seq 1 50001); do echo "abc4-$i" >> /tmp/lines4.txt; done
> >
> >     for i in $(seq 1 50001); do echo "abc5-$i" >> /tmp/lines5.txt; done
> >
> > fi
> >
> >
> >
> > loggernum=${1}
> >
> > if [[ -z ${loggernum} ]]; then
> >
> >     loggernum=10
> >
> > fi
> >
> > loopnum=$((loggernum/5+1))
> >
> >
> >
> > foreverloop='no'
> >
> > if [[ -n "$2" ]]; then
> >
> >     foreverloop='yes'
> >
> > fi
> >
> >
> >
> > while [[ 1 ]]; do
> >
> >     for i in $(seq 1 ${loopnum}); do
> >
> >         logger -p local0.3 -f /tmp/lines1.txt &
> >
> >         logger -p local0.3 -f /tmp/lines2.txt &
> >
> >         logger -p local0.3 -f /tmp/lines3.txt &
> >
> >         logger -p local0.3 -f /tmp/lines4.txt &
> >
> >         logger -p local0.3 -f /tmp/lines5.txt &
> >
> >     done
> >
> >
> >
> >     sleep 1
> >
> >     ps aux | grep logger | wc -l
> >
> >
> >
> >     if [[ "${foreverloop}" == "yes" ]]; then
> >
> >         echo 'wait for loggers to exit'
> >
> >         wait
> >
> >         journalctl_pid=$(pidof journalctl)
> >
> >         if [[ -z "${journalctl_pid}" ]]; then
> >
> >             echo "journalctl dead, exit"
> >
> >             exit 1
> >
> >         fi
> >
> >     else
> >
> >         exit 0
> >
> >     fi
> >
> > done
> >
> >
> >
> >
> >
> > 4) Wait for until the message "journalctl dead, exit" appears on the
> > console, this means journalclt exited.
> >
> >
> >
> >
> >
> > Rgds,
> >
> > --Ravi
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20210322/c99e90e4/attachment-0001.htm>