[systemd-devel] [PATCH] TODO: add various journal enhancements

[Kay Sievers]

On Thu, Apr 18, 2013 at 12:28 AM, Josh Triplett <josh at joshtriplett.org> wrote:
> On Thu, Apr 18, 2013 at 12:12:38AM +0200, Kay Sievers wrote:
>> On Wed, Apr 17, 2013 at 11:49 PM, Josh Triplett <josh at joshtriplett.org> wrote:
>>
>> > +  - Replace utmp, wtmp, btmp, and lastlog completely with journal
>>
>> We should definitely add the data needed to constuct this information,
>> if they are not already there.
>>
>> The tools could just use the journal directly, but there is the glibc api.
>
> Exactly; barring a compatibility library that provides the same
> functions as glibc, which seems hazardous, I'd like to support tools
> that only know how to read and write those files directly.

Hmm, from system's side I would consider the current state as
sufficient, and future work should probably focus on getting rid of
all that on systems that do not care, and not pimp up these crazy text
files.

>> > +  - Provide one or more FUSE filesystems:
>> > +    - Emulate utmp, wtmp, btmp, lastlog; read/write, recording credentials for all writes rather than limiting permissions
>>
>> That sounds like overkill. We should surely externalize the creation
>> of the files from systemd and make it optional to support legacy
>> stuff, otherwise these files should rather be phased out, than
>> emulated, I think.
>
> Separate, external, and optional, definitely; no sense running it if you
> have no tools around that read or write the format directly.  But given
> that the glibc API (and corresponding expectations about the file
> format) will exist forever, it seems sensible to have a way to support
> that without actually having the files on the filesystem.

I don't know what *forever* means here, I kind of doubt that. It's
pure legacy stuff that needs that, and that works with current
systemd. Most systems out there should not need these silly files at
all, and the use will surely not grow in the future.

>> > +    - Provide /var/spool/crash or equivalent, exposing an index of coredumps directly
>>
>> Coredump are really not consumed by any legacy thing that would need a
>> file system. What do you have in mind here. Specialized apps that need
>> that should just use the native API, not expect a file system, I
>> think.
>
> Just an idea for how to better satisfy user expectations.  Approximately
> zero cost, mounted on demand, gives a view into the journal.

Yeah, but it does come with a lot of costs, code, ressources, and all
just for one or two custom tools to consume that. I would avoid that.

>> > +    - Provide decoded text logs in /var/log
>>
>> I really don't think these plain text streams make much sense today.
>> If people want them, they should install syslog. Or some other project
>> can do that, I'm confident we do not want that code in systemd.
>
> Maintaining a separate copy of log messages seems like overkill; I want
> the data stored solely in the journal.  And I agree that the code
> doesn't belong in the journal itself; I'm suggesting that a standalone
> FUSE filesystem based on the journal library would prove useful for
> compatibility with both tools that want to consume the log format and
> users used to /var/log.  Seems pretty trivial to write, as well.

Yeah, sounds like a nice feature, but I'm not sure it belongs in
systemd. I don't want to manage logrotate on top of that fuse
filesystem, and all the other wild things that will happen then. Some
other project should open that can of worms. :)

>> > +  - Port upower to use the journal for historical power information used in future calculations
>>
>> Yeah, that would be useful.
>>
>> > +  - Support optional database-style indexes for frequent or performance-critical queries
>>
>> Well, it is not a database, it's a pretty specialized format based on
>> bisection lists. What specific thing do you have in mind?
>
> Telling the journal in advance that you'd like to make a lot of queries
> for PROPERTY, or for PROPERTY=value, and having it maintain an index
> that makes such queries faster than a linear scan and filter.

Hmm, I guess that works already just fine for the most common cases.
You've seen any real-world problems?

The managed data with the cursor support in the journal allow easy
live-export to real databases if such flexibility is required.

Kay