[systemd-devel] feature request: dlopen

[Luke Kenneth Casson Leighton]

On Wed, Feb 18, 2015 at 9:10 AM, Tobias Hunger <tobias.hunger at gmail.com> wrote:
> Hi Luke,
>
> I am mostly a lurker on the systemd mailing list, so my opinion does
> not carry weight in this community.
>
> On Tue, Feb 17, 2015 at 9:24 PM, Luke Kenneth Casson Leighton
> <lkcl at lkcl.net> wrote:> so i'm not going to "protest" - i'm going to
> try a different approach.
>> i'd like you to look at this list of debian packages that are
>> dependent on libsystemd0:
>> http://lkcl.net/reports/removing_systemd_from_debian/list_of_libsystemd0_dependent_packages.txt
>
> I understood most of these dependencies to be indirect: Packages that
> depend on other packages that in turn depend on libsystemd. Is that
> correct?

 that's right.  so, what that means is that the actual number of
packages which would need to be converted to dynamic loading is
actually very small (about 100), and the remaining 4,483 would be fine
(not need any work done on them at all).


> On the other hand the library is tiny and basically falls back to
> being a no-op in the case where systemd is not PID1, so it does not
> hurt non-systemd systems to have this library in any way.

 ...except that its introduction (usually --with-libsystemd) in those
100 (or so) packages has been done in a mutually-exclusive,
hard-compile-time switch that *excludes* the possibility of dynamic
(runtime) decision-making.


>> we see that a debian developer has created unofficial packages
>> that, if installed, provide replacements for key strategic packages
>> entirely recompiled *without* systemd and without libsystemd0 being
>> present.
>
> Good for them. I see very little value in replacing a ~150KiB library
> that does nothing for the users these packages are targeting, but
> everybody is free to spend their time however they want.

 they made it possible to remove systemd entirely.  they had to
recompile cups, stunnel, udisks2, upower, util-linux and many more...
here's the list:

  http://angband.pl/debian/dists/nosystemd/main/binary-amd64/Packages

 but here's the problem: because there is no dynamic run-time
decisions possible here, people are forced to adding that (unofficial)
repo and to risk their systems in the process.  reverting is also just
as risky.

 but, the worst thing is that because it's not an "official" repo, any
corporation that has for example a support contract is *prevented and
prohibited* from using the angband.pl repo because it would violate
their support contract.


>> moving on: in what adam wrote (rather hot-headedly, initially), he
>> goes on to mention that it would be perfectly reasonable to replicate
>> the effects of how he removed libsystemd0, in a way that would be far
>> less disruptive to end-users and sysadmins, and far less divisive:
>> dynamic library loading.
>
> Libsystemd's job is basically to provide exactly what you ask for: A
> wrapper around systemd functionality, that fails gracefully in case
> systemd is not used.

 honestly, i find it hard to argue with that.  i think i know what the
problem is.  the problem is, i believe, that the detection is not
user-controllable.  question: does libsystemd have a config-file
option that it reads, where if "systemd = disabled" is present, for
example, it is effectively equivalent to not having systemd installed?

 i'm going with the flow here, btw, even though it actually partially
undermines the case that i'm endeavouring to get across to the team.


> That wrapper is nicely packaged up into a library so that upstream
> projects do not need to keep reimplementing the same dlopen, error
> handling, etc. over and over again. Your proposal is to ask every
> upstream project to add that same snippet of code?

 yes.  in a dynamic way that includes a config file option which
allows run-time disabling of systemd.

>How about putting
> that into a library for easier reuse: Maybe libsystemdwrapper. That
> can then be wrapped in another wrapper when somebody freaks out about
> "everything is linking to libsystemdwrapper".

 haha yehh, it would look like that wouldn't it.  except that at the
first opportunity where it is configurable via a plain text file to
disable the chain-of-loading-loaders, the purpose would have been
achieved, so there would be no need for another loader-of-loaders.

 ok i'm going with the silliness, here, but you know what i mean.

> Maybe just renaming libsystemd would suffice? I am sure hardly nobody
> would object to having a tiny "libyzy" on their system:-)

 :)  in all seriousness, any kind of modification outside of what's
available from the distros is... it can't be done.  really.  this is
too low-level.  one mistake renaming a library without knowing the
consequences and people would end up with unbootable systems.  and
would be in violation of their support contracts. etc. etc.

>> so can i leave it with you to consider whether the current situation
>> is tolerable or not?
>
> Again: I can in no way speak for the systemd project. But from where I
> stand the systemd project went out of their way to provide you with
> exactly what you are asking for in a way that is easy to reuse by
> upstream projects. That is libsystemd. Apparently you find that
> solution objectionable, but I do not understand why.

 in many ways it doesn't matter that i (or anyone else) finds the
solution objectionable.  what should matter - to you - is that i (or
anyone else) wishes to make that choice [we might have a solution
above btw as you can see].

 the analogy here is libselinux1.  libselinux1 (which i've worked
with) can be disabled at boot time, even with a kernel option.  it can
even be re-enabled at runtime (if you made the correct option at boot
time).

 ... does libsystemd0 have that same user-configurable option?  or is
it implicit by way of whether systemd is PID 1 or not?

 .... i think... really, although technically what i am asking
*appears* to be "achieved in an equivalent fashion", they're not
entirely the same.

 bear with me whist i think this through, ok?

 my thoughts go like this: virtually every GNU/Linux distro has gone
for a mutually-exclusive all-or-nothing choice: systemd or out.  even
Debian has had to provide systemd-shim to quotes stop the whiners
quotes.  it has "Breaks: systemd < 209" in the package file.  they
*did not* notice that systemd may be easily disabled by not running as
PID 1.

 so, whole-sale, applications are *abandoning* the alternatives - in
droves.  there are discussions on the FreeBSD mailing lists about what
the hell they're going to do, because the code that *used* to be in
KDE5, for example, is being *ripped out*.  it's not even being
maintained as an alternative compile-time option.

so whilst it appears, on the face of it, that one might simply "not
run systemd as PID 1" and it would then be possible dynamically at
runtime for all the applications to go "oh, we're running sysvinit not
systemd, we should do XYZ rather than contact logind over d-bus"...

... in reality there is a massive and alarming near-total abandonment
of all the alternatives - all the incremental work done over the past
20 years - it's *all going*, making systemd the sole exclusive option
pretty much right across the board.

 that really really should alarm you quite dramatically.


> I would personally like to see a "libinitd" which brings the socket
> activation features that is provided to daemons as part of libsystemd
> to other init systems (that can support those). That would make it so
> much easier for upstreams to support more than one init system. But I
> would expect that to be implemented by the teams working on
> alternatives to systemd or by distributions centered around other init
> systems.

 i would _love_ to see a situation where there is more dialog and
collaboration between the developers of systemd and the maintainers of
the alternatives.

 and that i think, guys, is part of the problem.  the development of
systemd is progressing at such a fast pace, with no dialog with any of
the peers (developers of openrc, maintainers of sysvinit, depinit -
currently me - no-one) that there are people - really experienced
people -really freaking out right now, and they don't fully understand
why.

 they know that something's wrong, but are having a real hard time
putting their finger on it.  that's led to some er... charitably we
can call them debates ... that have resulted in really senior
dedicated people who have spend in almost all cases over a _decade_ of
their lives on debian... quitting their role and involvement.  and the
software libre community as a whole is seriously diminished by the
loss of their input.


>> i am one of the few people who can cut through all that, who has gone
>> to the trouble of digging into why libsystemd0 is found to be so
>> objectionable.  my take on the matter is that the technical arguments
>> - benefits or otherwise - of systemd and its alternatives - is
>> completely irrelevant.  over time people *will* develop alternatives
>> (and are already doing so: mdev, eudev, uselessd, openrc and many
>> more).
>
> Sure. I am looking forward to that! I am convinced a bit of
> competition and fresh ideas will do systemd a hell of a lot of good:-)

 the problem is, systemd conversion is so near-exclusive and so rapid
that there won't *be* any competition.

 the "competition" needs to have a user-base on which to progress and
be developed.  even a small break in continuity would mean, instantly,
bit-rot that would make it incredibly hard to reinstate the
alternatives.

 that should have the systemd team very very alarmed.  that nightmare
scenario where you will *have* no major competition is happening RIGHT
NOW.

 gentoo are about the only (major-ish) GNU/Linux distro that's even
remotely contemplating providing alternatives, and the only reason
they may consider doing so is down to the unique nature of their user
and developer base: users are happy to do near-complete or complete
recompiles of the entire code-base (!).

 centos: going with the flow of fedora.
 debian / ubuntu: converted.  abandoned everything but systemd.  going
with the flow.
 slackware: minimising workload (staying on the other side of the fence).
 puppy / mint: going with the flow of ubuntu.

>> no, i feel that it really does have nothing to do with the technical
>> benefits of the available options: what people are finding completely
>> objectionable is that they have *no good choices*.  it's "use systemd
>> or go away" - and unfortunately almost without exception (slackware
>> and FreeBSD being two notable ones) that "piss off" attitude is being
>> replicated across *the entire GNU/Linux Distro world*.  the situation
>> is completely unprecedented and without parallel in the short history
>> of software libre (and that's something that, honestly, i find to be
>> really shocking, hence why i am contacting you).
>
> My take is a bit different: I have seen and used lots of init systems
> over the years. *Finally* we have one that actually provides some
> benefits that developers of unrelated projects actually want to use!
> That none of the others ever got to the state is more a testimony for
> their failure than for their design -- even if many loud-mouthed
> systemd opponents seem to think otherwise.

 yehhh.. we could get into that.  we could get into various technical
analyses of the benefits of each, and in fact many _many_ people have
done exactly that... i've read as many of them as i can stand [if i am
honest about it :) ]

 ... and i feel that all of the analyses miss the point, which is,
primarily, that the rapid [and accidental] exlusionary adoption of
systemd right across the board is destroying - almost wholesale - the
alternatives.

 with no userbase, they won't *have* user or developer mindshare by
which continued work can be justified to consider including them in
any major distribution.


> Please do not ask systemd to be less useful, please ask other projects
> to implement better (for whichever metric of "better" you want to
> apply) solutions to the problems those developers face.

 i'm not asking systemd to be "less useful", i'm asking the team to
take stock of the situation, to be aware that there isn't going to be
anything *other* than systemd for anything but the most obscure
(mostly source-compiled) GNU/Linux distributions such as openwrt,
openembedded, gentoo and so on.

 which leaves anyone who feels that the way that they use and maintain
their current GNU/Linux distro (be it desktop or server) is better off
not having systemd *despite* the advantages of systemd, being
completely and utterly screwed.

 and... betrayed, if i am absolutely honest.



>> overall, they feel that they're being forced into the use of something
>> that they feel has not been properly thought through, is still under
>> development, is increasing in scope in a way that alarms them due to
>> there being no other choices, causes them some huge inconvenience that
>> they'd rather have a bit more time to consider but they are *not being
>> given that chance*, and much more.
>
> I do not see how the systemd project or anybody else can change that
> at this point.

 well, apart from saying "really, you should have listened more
carefully before rushing ahead" - a statement which doesn't help fix
the problem *now* but may help others in the future (reading this on
archives) to avoid a similar situation (one might hope), i feel that
it really is down to everyone to work out how to make sure that other
init systems can be included - at runtime - before it really _is_ too
late.


> My experience is that many people out there are beyond
> a rational debate at this point.

 they've only become irrational because they felt like they were being
ignored or betrayed.  and this really is quite a complex subject that
many perfectly rational people have been... simply unable to
comprehend [i'm good at analysing cross-project dependencies and
issues, and even i am having difficulty getting to the bottom of what
the hell is really going on here].

 the other issue i feel is that these are technically-minded people,
usually who have a charter that says they have to prioritise and
respect *technical* decisions (the Apache Software Foundation Charter
is one), and there's simply no precedent - on such a wholesale
distro-wide scale, by which they can guage what to do.

 so they try their best but honestly it's so complex, and so out of
their experience, that, sadly, they get into severe arguments and we
end up with good people leaving the projects that they love, no longer
wishing to speak to people that they have worked with in some cases
for nearly 2 decades of their lives.

 i have to say it but this is by far and above _the_ worst situation
that the software libre community has ever faced.  all other bust-ups
resulting in forks or new code being created are nothing compared to
this, because it's an "all-at-once" wholesale conversion that's
streaked across all the major distros.


> And I explicitly want to include
> people from both sides of the fence in this statement.

 ... yeah.  i've read enough to know that's very true.  the key here,
after talking to a lot of people, i think is that this really isn't a
"technical merit" issue, it's a strategic one across *all* the
GNU/Linux distros.

 ... do you know of any one person who is authoritatively making
strategic decisions across all of the GNU/Linux distros?  ...
rhetorical question btw... :)


> I am afraid we will have to sit this out.

 sitting this one out is only going to make matters worse.  you're on the clock.


>> i have to tell you: i even heard, on slashdot, that microsoft is now
>> using - to significant success - the situation surrounding systemd as
>> a sales pitch to have GNU/linux systems successfully replaced with
>> windows servers.
>
> Isen't it amazing what kind of stories some anonymous cowards make up
> over at slashdot? There are some gems of creativity in some of those
> systemd flamefests.

 amazingly the one i started still has, a week later, a few comments
being added (750 and climbing).  it also differs in that the comments
*are* quite rational and insightful if they're marked as "insightful"
by moderators.  even the flame-fest individuals served a really useful
purpose of allowing saner minds to make sensible contributions through
contrast.

l.