[systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance

[Colin Guthrie]

Martin Steigerwald wrote on 21/10/14 10:25:
> Am Mittwoch, 8. Oktober 2014, 10:54:00 schrieb Lennart Poettering:
>> On Tue, 07.10.14 23:40, Uoti Urpala (uoti.urpala at pp1.inet.fi) wrote:
>>> On Tue, 2014-10-07 at 14:15 -0400, Rob Owens wrote:
>>>> My question really isn't "why are the Debian dependencies the way they
>>>> are".  I understand that.  I was trying to highlight the strange
>>>> situation of a desktop application requiring a particular init system. 
>>>> I *think* this is a result of the init portion of systemd being bundled
>>>> together with the logind portion of systemd.  To me (unfamiliar with
>>>> the systemd code) those two functions seem distinct enough to merit
>>>> being separate.  Debian can't easily separate what the systemd devs
>>>> have developed as a single binary, so we end up with these strange
>>>> dependency chains.> 
>>> "Single binary" is false of course. Logind is developed as a separate
>>> program, which is why systemd-shim is possible at all.
>>>
>>> AFAIK the actual relevant dependencies go as follows: First, there's a
>>> good reason why logind requires cgroup functionality. And there's a good
>>> reason why cgroup functionality is best implemented together with init
>>> (see
>>> http://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/
>>> for more info). So it's not quite directly "logind has to depend on
>>> systemd as init", but "logind has to depend on the system having cgroup
>>> support, and there's no equally good cgroup support available for inits
>>> other than systemd". It is possible to provide the relevant cgroup
>>> interfaces in or on top of another init, as the systemd-sysv + cgmanager
>>> combination attempts to do. But it is not trivial to do, as bugs and
>>> implementation delays with that combo have shown, and it's quite likely
>>> that there will be more problems in the future. It's not a particularly
>>> good idea to use the less-tested and less-reliable systemd-shim instead
>>> of the more reliable systemd. Thus the overall result is that yes, it
>>> does make sense to switch machines to systemd when you add certain
>>> functionality, even if that functionality does not appear to be directly
>>> tied to the init system at first glance.
>>
>> Also note that the systemd concepts logind makes use of are also
>> exported in its own API. The "scopes" and "slices" that logind uses
>> are exposed on its bus objects, and they are used by tools like
>> "loginctl" to do their work.
>>
>> The "scopes" and "slices" concept does not exist elsewhere, and
>> there's nothing comparable around, so even if we wanted we couldn't
>> make logind work on anything else.
> 
> Then why in the first hand are the "scopes" and "slices" concepts within 
> systemd *tightly* coupled when it is functionality that makes sense to be 
> utilitizes in a component that provides a different functionality.
> 
> I wonder what functionality systemd provides right now in one more or less 
> tightly coupled package:
> 
> - early boot process

PID 1

> - managing kernel drivers and device files (via coupled udev)

Not PID 1

> - resource control for process groups

PID 1.

> - logging

Not PID 1

> - core dumping

Not PID 1

> - network while I think a distro-unified way to approaching network that 
> replaces the current distro specific ways like ifupdown and so on, why does it 
> have to be coupled with systemd?

Not PID 1

> - cron, although that at least in Debian is separate as systemd-cron

Partly PID 1

> Thats rather much.
> 
> Really rather much.
> 
> Way more than traditonal sysvinit covered.

This is because traditional sysvinit covered PID 1 and performing it's
job (if you consider e.g. killall5 and friends).

You seem to be conflating systemd as PID 1 and systemd as a project.
These are two completely different things.

There are many related things here and a lot of the sub components
obviously use a lot of the same functionality and utility functions.
This is achieved via a shared library with a private API.

The only reason that all these separate parts are developed as part of
the systemd project is that it's *easier*. It's just the easiest way to
develop.

An alternative would be to make the utility functions API stable and
export the shared library publicly and give API guarantees, but that
puts a lot of pressure and it's a difficult thing to provide and it has
long term maintenance overhead. This is something that *costs*. It costs
in time/man hours and therefore it carries real overheads. Doing this
for the convenience of splitting things out is simply not worth it -
especially so as the main people working on these things are the same
people.

It also increases the test matrix. If logind v300 has to work with
libsystemd v300 and all future versions then the testing side of things
increases in complexity *massively*. Again this causes problems that
translate to time and effort of developers that could better be
allocated to building a better overall set of building blocks.


The other option would be to just split out the git repos but keep the
API a private one - with releases still in lockstep. This is ultimately
*no* different to keeping it all in one repo but gives an outward
impression of separate components. Doing this would be purely political
and would provide no technical benefit. It would also cost. It would
cost because developers would have to post patches to separate patches
to the mailing list for review. This makes both git send-email and git
am harder to manage and is just pure avoidable overhead. It also pushes
a lot of additional effort to downstreams where all the separate
components still have to be updated at the same time.

So overall, keeping things developed as one project is a technical
decision that has real benefits both for the developers doing the work
and the downstreams consuming it. Changing the approach would be 100%
political and, quite frankly, the systemd developers have very little
time for such games. They just want to get on and build better systems.

> What of this actually *needs* to be coupled with systemd? What of this needs 
> to be coupled tightly to Linux kernel? 

If people want to provide implementations of such components outside of
systemd they would be more than welcome to do so.

As systemd provides a lot of the functionality and shared API functions
that are convenient to use (i.e. the base building blocks) and as people
who are working on the systemd project are doing the actual work, and as
these are seen as core building blocks of a modern OS, then it makes
sense to do this under the systemd umbrella.

There is no conspiracy here. It's just convenient. If other people want
to do this work, great.

> systemd is driving a road where its all of this functionality by default is 
> only available for Linux and where upstream said they just won´t accept 
> patches – is that still true? – for portability. 

systemd developer hs have stated this clearly several times over, but to
say it again, it's not that systemd *won't* accept patches for
portability, it's just that there will not be a codebase littered with
#ifdefs and optional codepaths that are not well tested by the core devs.

systemd makes use of APIs that are currently *only* available on Linux.
If other kernels implement the same APIs, then systemd will work with
them just fine. If they implement equivilent functionality under
different APIs, then it's unlikely the systemd developers would support
this upstream directly as this would result in these #ifdefs and
untestable paths (by the core developers) that are very unattractive for
the upstream project.

But you know what? This doesn't matter! Those developers who are
interested in those other kernels will just maintain their own fork of
systemd (and by form I mean a true fork - one which git facilitates well
where upstream changes can be merged periodically) where the linux calls
are split out into $whatever calls.

This separate project can be tested by those developers and they can act
as the first port of call for bugs in their version but collaborate with
the original systemd for issues that are present in their upstream. In
many cases the developers of this fork would have upstream commit access
to. Such a world is perfectly feasible and it allows systemd to keep a
clean codebase and the downstream too. This is one of the cases where
committing everything upstream is not beneficial to either systemd or
the fork. Both are more individually readable and maintainable by their
relative maintainers. Everyone wins.


Having everything "committed upstream" is not necessarily the holy
grail. There are good, solid technical arguments not to work that way
and yet still get on and work well.

Forking and doing stupid things like renaming functions for the sake of
it, or changing commenting or code style is obviously something that any
well intentioned forker would not do. Not everyone will agree with
everything in this sphere but everyone can adapt and get on. That's just
part of working on FOSS projects.


> systemd provides more and 
> more functionality that desktops like to use, that other tools like to use.
> 
> When Microsoft back then did something like this it was called "Embrace, 
> Extend and Extinguish"¹…
> 
> a) Embrace: systemd provides functionality that is mostly compatible or 
> similar with what sysvinit and other components provided
> 
> b) Extend: systemd extends on that functionality and makes it more and more 
> difficult for desktops and apps to ignore that kind of functionality offers
> 
> c) Extinguish: The scope of systemd becomes so broad, the functionality it 
> offers in a – admittedly – convenient package to often needed and used, that 
> other ways of providing the functionality are extinguished.
> 
> Really… it matches quite closely.

Oh come on! This is just an attack and FUD. You make repeated claims of
coming in good faith etc. and seem to dismiss any technical defence
being made with vague references and then you bring out a aggressive and
argumentative statement like the above.

Again, you're totally confusing systemd as a project and systemd as a
PID 1. You need to accept and understand that distinction before anyone
here will take you even remotely seriously.

> I don´t think this was your plan. But… when looking at the currently visible 
> outcome this is quite exactly what is happening here.

No. Looking at the visible outcome of this to me is that huge progress
is being made in documented APIs that solve real world projects backed
up by solid, supported and maintained implementations. To me this is a
great thing. You are reading into it what you want based on your
preconceptions and then building up a case based on that. This is really
a case of apophenia, but people just don't see that because they are so
blinkered by their prejudices.

> So if you can follow this just a tiny bit: Do you start to understand why 
> systemd triggers the uproar, polarity and split so obvious that one needs to 
> have hands before both eyes for not seeing it?

What do you expect? Ultimately, haters gonna hate. There are a group of
people who are predisposed to hating change. systemd replaces a lot of
core functionality. If that was rolled out *without* causing a group of
people to complain I would have been surprised.

Of course this criticism is listened to and often actions are taken
because of it, but what do you expect the outcome to be? Do you expect
all the repos to be split up? Stable APIs exported and supported? What
outcome do you actually *want* here? You seem to be doing lots of
complaining, but very little in the actual suggesting what could be done
different that has not already been addressed technically. You may
disagree about that end decision but that's just the way it is
sometimes? The people doing the work ultimately get to make the decisions.

> So as I see it: there are *real* concerns. And if systemd is meant to be a 
> tool for users and admins, I *highly* and *strongly* recommend that you as 
> systemd closely look at these concerns. Unless you are willing to trigger the 
> polarity, the split and the uproar that it creates.
> 
> In KDE more and more voices come up that developers need to talk to their 
> users. Sure… its a do-ocracy. But still… if you want to become really broadly 
> accepted and successful with something you offer… its important to listen for 
> feedback. And in my oppinion you are in a good opportunity here, cause I think 
> there is no lack on feedback on systemd.

You also seem to think that this feedback is somehow special or new.
Such feedback has been coming in for years and much of it has already
been listened to and acted on or decided as something not to be
supported or irrelevant. All of your posts so far seem to be worded like
you think this process has not happened at all. There is a wealth of
archive information, blog posts, conference talks and such like out
there to document this process. You need to be proactive to consume it
all, and you cannot expect to be spoon fed it all.


Honestly, I've been following this project since it's very inception and
I from what I see your opinion of it has been formed around notions that
are pretty much entirely fictional or based on second hand
information/FUD. The accusations you make are often wildly technically
inaccurate and you seem to offer no credit to the devlopers with regards
to their collaboration.

I've been doing the rounds on mailing lists, distros and conferences for
years. systemd is one of the most open projects with regards to doing
the conference circuit and talking to people - all over the world and
with interested parties both commercial and community run. There is no
end of opportunity to speak with the systemd developers - often in
person - and discuss the issues. The regurgitation of the same old stuff
on mailing lists today no longer warrants further discussion for the
most part. Doing so just soaks up developer time (and it takes a *lot*
of time to reply to these things - this email alone has likely taken me
20mins+ - I'd much rather developers give such "feedback" a brief read
over for any salient points and then ignore it and get on with designing
and writing code and making a difference.

Sorry, but I just do not see things they way you do and your repeated
regurgitation of factually incorrect statements is doing nothing but
making you appear to be someone to not waste time replying to further.
This is not ignorance or not listening to feedback or anything of the
like... it's ultimately, I'm sorry to say, just not feeding the
trolls[1] - or, if you prefer a more sanguine version, not replying is
just agreeing to disagree.

Col

1. for the avoidance of doubt, I don't mean anything nasty by this - I
just mean replying just continues a pseudo "discussion" with someone who
is not in any way listening to your arguments or accepting your
technical opinion - it's just a never ending discussion that leads
nowhere - it has to be starved to die off and cannot be fed.


-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/