[systemd-devel] jio test results (was: [ANNOUNCE] jio is an experimental systemd-journald journal file tool utilizing io_uring)

[Vito Caputo]

On Thu, Nov 26, 2020 at 12:28:16PM +0100, Paul Menzel wrote:
> [No idea, if I should carbon copy the list or not.]

I hope you don't mind, but I've CC'd the list in replying.

At this point it's helpful for others to see:

1. jio compiled and worked
2. jio didn't crash or rm -Rf / your system
3. what kind of information it produced

> 
> Am 26.11.20 um 04:02 schrieb Vito Caputo:
> > Hello systemd-devel,
> > 
> > Recent discussion here about journal space consumption happened to
> > occur while I was exploring use of the new io_uring linux kernel
> > interface in combination with journal files.
> > 
> > What began as a curiosity about this new kernel interface, and what it
> > would be like to program journal file processing with some kind of
> > continuation-passing style C code bolted onto it, evolved into
> > something already providing new visibility into journal-file space
> > utilization not AFAIK currently offered by systemd's journalctl (not
> > that it couldn't be added there too).
> > 
> > I've called this program jio, pronounced "jai-oh".
> 
> Thank you for writing and sharing this.
> 

You're welcome, it was fun to write. Thanks for being a guinea pig!

> > At this time jio implements three basic functions:
> > 
> > 1. `jio report usage`
> > 
> >      READ-ONLY
> > 
> >      Measures and reports space actually used by objects in all
> >      accessible journal files, classified by object type.
> > 
> > 
> > 2. `jio report tail-waste`
> > 
> >      READ-ONLY
> > 
> >      Measures and reports unused space allocated to the tail ends of
> >      all journal files, classified by journal state.  Journal state
> >      being Online, Offline, or Archived.
> 
> Thank you. It builds fine on Debian Sid/unstable with GCC 10.2.0-19 (some
> warnings), and the results are attached.
> 

Any chance you could mail me the warnings?  I see none here, would
prefer a clean build for others.

<snip>

> Per-object-type usage:
>           UNUSED: [0] 0.00 B
>             Data: [2404148] 261.85 MiB
>            Field: [6491] 331.51 KiB
>            Entry: [1521426] 712.38 MiB
>    DataHashTable: [100] 355.56 MiB
>   FieldHashTable: [100] 521.88 KiB
>       EntryArray: [554339] 577.14 MiB
>              Tag: [0] 0.00 B
> Aggregate object usage: 1.86 GiB of 2.22 GiB spanning 100 journal files

> 

 <snip>

> 
> Totals:
> 	Tail-waste by state:
> 		   Offline [3]: 7.23 MiB, 2% of all tail-waste
> 		    Online [1]: 7.37 MiB, 2% of all tail-waste
> 		  Archived [96]: 339.82 MiB, 95% of all tail-waste
> 
> 	Aggregate tail-waste: 354.42 MiB, 15% of 2.22 GiB spanning 100 journal files

This is useful information.  On my system there's an accumulation of
Offline user journals accounting for a majority of the tail-waste,
which being Offline the `jio reclaim tail-waste` won't touch.  We need
more samples from the field, but this suggests it might be something
unique to my system.

Thanks again!