[systemd-devel] journald forwarding to rsyslogd. Huge (350 times) performance degradation. What am I doing wrong???

[Reindl Harald]

Am 20.09.20 um 16:09 schrieb Vitaly Repin:
> Thanks for the answer!

please avoid off-list copies and top-posting when you got an answer with
proper quoting (you break threading and produce unreadable conversations
by permanently switch between quoting styles)

> However, it looks like that imjournal module is not an option for me.
> 
> rsyslog imjournal module documentation
> (https://www.rsyslog.com/doc/v8-stable/configuration/modules/imjournal.html)
> clearly recommends to use imuxsock module in my case:
> 
> [begin quote]
> As such, the performance of a configuration utilizing this module may
> be notably slower than when using imuxsock.
> The journal provides imuxsock with a copy of all “classical” syslog
> messages, however, it does not provide structured data. '
> Only if that structured data is needed, imjournal must be used.
> Otherwise, imjournal may simply be replaced by imuxsock, and we highly
> suggest doing so.
> [end quote]
> 
> I do not need structural data and I need performance. My system
> generates a substantial amount of logs. I want to be able to store
> messages with an arrival rate of 10 000 messages per second at least.

with "Storage=volatile" there is is not space for "database corruption"
and given that it *reads* the journal on that channel is not much space
for ratelimiting unless you hit the ratelimits of journald itself

[root at srv-rhsoft:~]$ cat /etc/systemd/journald.conf
[Journal]
Storage=volatile
RateLimitInterval=1s
RateLimitBurst=20000
RuntimeMaxUse=100M
ForwardToSyslog=no

>> Max rate at which log messages are NOT dropped is 17250 msg/s for logging to /run/systemd/journal/syslog.  And only 47 messages per second for logging to /dev/log.
> 
> 
> Den sön 20 sep. 2020 kl 12:26 skrev Reindl Harald <h.reindl at thelounge.net>:
>>
>>
>>
>> Am 20.09.20 um 12:07 schrieb Vitaly Repin:
>>
>>> ForwardToSyslog=yes
>>
>> this is long obsolete
>>
>>
>> $WorkDirectory /var/lib/rsyslog
>>
>> module(load="imjournal" StateFile="imjournal.state"
>> WorkAroundJournalBug="on" Ratelimit.Interval="600" Ratelimit.Burst="50000")