[systemd-devel] [EXT] Re: Changing database formats for systemd-journald

[Windl, Ulrich]

Well,

I had almost 3 minutes, and someone else from SLES support testing hat 20 minutes to read the logs on SSD. SLES support was able to speed up thing by a factor of 500 or so. However I don’t know what they did. I guess the update will be published soon.

Kind regards,
Ulrich Windl

From: systemd-devel <systemd-devel-bounces at lists.freedesktop.org> On Behalf Of Mantas Mikulenas
Sent: Saturday, June 28, 2025 11:49 AM
To: Gunnar Guðvarðarson <gunnar at meh.is>
Cc: systemd-devel at lists.freedesktop.org
Subject: [EXT] Re: [systemd-devel] Changing database formats for systemd-journald


Sicherheits-Hinweis: Diese E-Mail wurde von einer Person außerhalb des UKR gesendet. Seien Sie vorsichtig vor gefälschten Absendern, wenn Sie auf Links klicken, Anhänge öffnen oder weitere Aktionen ausführen, bevor Sie die Echtheit überprüft haben.
On Thu, Jun 26, 2025 at 6:27 PM Gunnar Guðvarðarson <gunnar at meh.is<mailto:gunnar at meh.is>> wrote:
I have attempted to use systemd-journald as a centralized log collector, it works fine and the interface provided by journalctl is very convenient to quickly find things, following logs and other things.

But I quickly discovered the performance implications of doing so, as when the journal gets over a certain size, it practically ceases to function.

See https://github.com/systemd/systemd/issues/2460 for details.

"Takes 50 seconds to display logs" is slow, yes, but that's nowhere near the same as "practically ceases to function"?


This makes rsyslog's omjournal, and systemd-journal-remote basically useless, right?

So, serious question, can the journal be migrated to a better (indexed?) database format, for example SQLite?

Maybe it could, but journal entries do not have fixed columns (except for timestamps and seqnum), so you wouldn't have one neat SQL table of entries – you'd probably end up with a large <id, fieldname, value> "pile of attributes" table and I'm not sure if that would perform any better. (The current format *is* indexed – each field has its own index, not just specific ones.) I think the journal is closer to a "wide-column store" or "NoSQL database" than a traditional SQL database? I'm not even close to being knowledgeable in this topic, though.

But I'm not sure if a format change would help on HDDs, anyway... I have a small project that involves a ~10 GB SQLite database on an HDD, and it too sometimes takes minutes to perform the first query from cold cache (long enough that I had to extend the web server's request timeouts).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20250630/f6f47c6b/attachment.htm>