[systemd-devel] [PATCH] preset-transient

[Dimitri John Ledkov]

On 6 February 2015 at 17:29, Didier Roche <didrocks at ubuntu.com> wrote:
> Le 05/02/2015 17:11, Dimitri John Ledkov a écrit :
>>
>> Some context for this patch.
>
>
> Hey Dimitri, thanks for working on that. I'm just giving a broader context
> for everyone who followed the past discussion we had in december/january.
>
> This is a followup on our discussion and what we discussed on the "/usr vs
> /etc" email thread (to give the full context).
>>
>>
>> I would like to support a new preset model, which has the following
>> properties:
>>
>>   - distribution shipped defaults are enabled
>>   - and are applied to each boot/upgrade
>>   - without overriding any user configuration
>
> Said differently, the Debian/Ubuntu needs are that distro defaults are
> migrated. We would prefer an upstream solution for this than the current
> cache implementation by our install helper tools that Dimitri described
> below.
> We should be able to handle defaults per flavors in the ubuntu case
> (meaning, alternatives are respected, with a priority order based on
> filename), admins should be able to disable services that are enabled by
> default on the distro and admins overrides are always respected, even if the
> distro changed its default policy for a service from disabled to enabled or
> the contrary.
>
> We need for this to work the /dev/null symlink in /etc patch to disable
> known services that was discussed in the previous thread and agreed on.
>
> From the past discussions and during the hackfest, we agreed that presets
> were the way to go forward, but with slight changes that Dimitri explains
> below.
>
>>
>> In many ways it is very similar to existing functionality but not
>> quite possible to achieve all of the above.
>>
>> Thus, I'm introducing a new optional functionality, new unit
>> configuration directory, and new transient-preset configurations.
>>
>> On each boot, if TransientPreset=yes, presets from
>> /usr/lib/systemd/system-preset-transient/*.preset are applied into
>> configuration path /run/systemd/system-preset-transient/.
>
>
> Hum, we told at the sprint that we wanted to be that available for everyone,
> and not having any conditions. Distros which still desires only the existing
> behavior would not ship files in *-preset-transient directories.
>

Running any presets commands unconditionally, without any preset
configuration is harmful.
Prime example, if one doesn't ship any .preset (even the one that
comes with systemd) shutdown.target gets activated and all that system
does on boot is going to shutdown straight away.
Thus anything but presets that end with "disable *" is potentially
quite harmful - or presets/transient-presets that are in concert.

Install time preset, at the moment is executed when machine-id is
missing (Aka condition first boot). Which imho is a weak declaration
rather than explicit one. If either of the presets are not configured
properly, one shouldn't run them. And being explicit about them is
long term good. See more about it below.

>>
>> An upgrade tool, sysadmin can repeat that action at appropriate points
>> by also calling: systemctl --runtime preset-all.
>
>
> This command is only for sys admins, on package upgrade/installation/removal
> (having units file or a new preset file), we would call as a trigger
> systemctl --daemon-reload, which then, would purge the transient runtime
> directories and reapply needed changes.
>
>>
>> If distribution integrates usage of Transient Presets, it gains a few
>> very nice properties. Fresh installations, much upgrades. User/admin
>> modifications are preserved. And there is no additional logic required
>> to maintain separation / diffs between system-defaults and
>> user-modifications. At the moment distributions like Debian (where
>> most things are enabled by default) maintain a complex state in /var/
>> which tracks which things were distro-enabled before/after the
>> upgrade, as well as whether user/admin has disabled/enabled things
>> before/after the upgrade and try hard to correctly reconcile the
>> correct state for all units. However, with this patch, most of this
>> segregation moves away.
>
>
> We discussed first that this could have gone in /var (this transient layer
> state is under our control) so that it's not called at every boot, however,
> /var isn't required at this very early stage to load units from systemd.
> We would like to not add those in another /etc directory on the broader goal
> of "empty /etc by default".
>>
>>
>> The remaining part, which is not addressed in this patch series, yet,
>> is the ability to override .wants/ symlink from a higher order
>> configuration directory. That is if the following symlinks are present:
>>   /etc/systemd/system/foo.service.wants/bar.service -> /dev/null
>>   /usr/lib/systemd/system/foo.service.wants/bar.service -> ../bar.service
>> There is no wants dependency added from foo.service -> bar.service.
>> This bit is discussed in details and agreed upon on the mailing
>> list. (Unwants thread has urls to the messages)
>>
>> Regards,
>>
>> Dimitri.
>>
>> Dimitri John Ledkov (1):
>>    Add support for transient presets, applied on every boot.
>>
>>   man/systemd-system.conf.xml |  1 +
>>   src/core/main.c             | 30 +++++++++++++++++++++++
>>   src/core/system.conf        |  1 +
>>   src/core/unit.c             |  2 +-
>>   src/shared/install.c        | 59
>> ++++++++++++++++++++++++++++++---------------
>>   src/shared/install.h        |  2 +-
>>   src/shared/path-lookup.c    |  2 ++
>>   7 files changed, 76 insertions(+), 21 deletions(-)
>>
>
>
> Right now, I think that we shouldn't have a configuration flag for it, this
> should apply (as stated above) to any setup, and distros can opt in or out
> just by shipping those transient preset files.
>

Well, I understood, that we should allow both distros and user to
opt-into either or both. Plus to successfully use transient-presets,
one needs regular presets configured in concert as otherwise things go
haywire as you have noted.

With this patch to use transient-presets at the moment one needs to:
ship "disable *" system-preset
ship preset policies in transient-presets
ship /usr/lib/systemd/system.conf.d/distro.conf -> which has
[Manager]
TransientPreset=yes

This way on installations were distribution does not ship nor enable
transient presets, one can still use them adhoc by dropping transient
.preset files in /etc and also enabling TransientPreset option in
/etc/systemd/system.conf


> It seems to me that this code has some flaw on first boot: As no preset file
> (not in the transient directory) is comparable to "enable *", that would
> mean that on a freshly installed system (without a /etc/machine-id file),
> systemd is going to apply systemctl enable on all services before the
> transient preset, and thus, will create enablement symlinks in /etc for
> every services available with an Install stenza on the system.
>
> I guess the condition should rather be: have an implicit enable * for
> permanent preset if there is no transient preset files detected.
>
> Does it make sense?
>

Making auto-detection of whether to apply presets or not at the early
startup to be based on cascading configuration directories is fragile,
as one would need to stat/readdir all configuration directories:
/usr/lib/systemd/system-preset-transient, /usr/local, /etc. But even
that will not be enough, as one would then need to check those preset
files to make sure they are really empty, and don't just have
commented out and blank lines. That's a lot of overhead. Given that
current presets check a simple stamp file, and load everything lazy
(well all units are iterated, but the preset files are consulted
lazily until first match is found)

Excluding one presets type, depending on whether the other one is
configured is confusing and not very additive. Especially if one wants
to opt-into the other type of presets, despite distribution
integrating only one type.

Would it be more consistent if I add "Presets=yes" option, (enabled by
default for backwards compat) which will guard and not apply regular
presets even if machine-id is missing? That way, e.g. debian-like
distro could ship:
/usr/lib/systemd/system.conf.d/distro.conf ->
[Manager]
Preset=no
TransientPreset=yes

Which is safer than fiddling everywhere for /etc/machine-id to be
available (granted that has already been done in most places)

> Another opened question: (to me, the answer should be yes): should we have
> user-transient-preset as well so that we can have the same behavior and
> options for user-level systemd management.

I believe we should and there is barebones support for some user
presets already. However I have noticed that it seems like traditional
user presets are not applied at all. It wouldn't make sense to
generate / apply them at early stage of pid's 1 main(), so maybe they
are hidden somewhere else (E.g. PreStart of user at .service unit would
be a good place, but seems to be absent).

-- 
Regards,

Dimitri.

Intel Corporation (UK) Ltd. - Co. Reg. #1134945 - Pipers Way, Swindon SN3 1RJ.