[systemd-devel] Changing database formats for systemd-journald

[Mantas Mikulėnas]

On Thu, Jun 26, 2025 at 6:27 PM Gunnar Guðvarðarson <gunnar at meh.is> wrote:

> I have attempted to use systemd-journald as a centralized log collector,
> it works fine and the interface provided by journalctl is very convenient
> to quickly find things, following logs and other things.
>
> But I quickly discovered the performance implications of doing so, as when
> the journal gets over a certain size, it practically ceases to function.
>
> See https://github.com/systemd/systemd/issues/2460 for details.
>

"Takes 50 seconds to display logs" is slow, yes, but that's nowhere near
the same as "practically ceases to function"?


> This makes rsyslog's omjournal, and systemd-journal-remote basically
> useless, right?
>
> So, serious question, can the journal be migrated to a better (indexed?)
> database format, for example SQLite?
>

Maybe it could, but journal entries do not have fixed columns (except for
timestamps and seqnum), so you wouldn't have one neat SQL table of entries
– you'd probably end up with a large <id, fieldname, value> "pile of
attributes" table and I'm not sure if that would perform any better. (The
current format *is* indexed – each field has its own index, not just
specific ones.) I think the journal is closer to a "wide-column store" or
"NoSQL database" than a traditional SQL database? I'm not even close to
being knowledgeable in this topic, though.

But I'm not sure if a format change would help on HDDs, anyway... I have a
small project that involves a ~10 GB SQLite database on an HDD, and it too
sometimes takes minutes to perform the first query from cold cache (long
enough that I had to extend the web server's request timeouts).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20250628/5fa524b6/attachment.htm>