[systemd-devel] Journald retaining logs for only 10 days

[Vito Caputo]

On Sat, Nov 14, 2020 at 09:31:23AM +0000, Nikolaus Rath wrote:
> Hello,
> 
> I just discovered that on one of my systems journald only retains log
> entries for about 10 days:
> 
> # journalctl | head -1
> -- Logs begin at Wed 2020-11-04 15:57:13 UTC, end at Sat 2020-11-14 09:28:19 UTC. --
> 
> I do not understand what could cause this, because I have no retention
> limit configured, and the logs take up way less space than I have
> reserved:
> 
> # journalctl --disk-usage
> Archived and active journals take up 320.0M in the file system.
> 
> # journalctl > alllogs
> # ls -lh alllogs 
> -rw-r--r-- 1 root root 27M Nov 14 09:24 alllogs
> 
> 
> Can someone help me understand where the log entries have gone?
> 
> # journalctl --version
> systemd 241 (241)
> +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid
> 
> # grep -vE '^#' /etc/systemd/journald.conf 
> 
> [Journal]
> SystemMaxUse=300M
> 
> 

One thing to consider is journald allocates space per-file in 8MiB
increments.

On my laptop for example, there are 27 user journals, 8MiB each, where
the last object offset is around 2MiB.  This alone burns ~162MiB in
allocated but unused space.

We should probably have some lower level tooling for scrutinizing the
journal files and reporting how much of the space is actually used vs.
fallocated.

Regards,
Vito Caputo