[systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance

[Martin Steigerwald]

Colin,

I had the feeling that is a bad idea to read your mail before I go to sleep. 
But I was interested in what you have to say since you made quite an effort in 
your reply to me. And now I can´t sleep since my head if full of thoughts and 
I am full of emotions as well.

With that I perceive starts an answer on a technical matter ends with what I 
received as a dire personal attack: I.e. calling me names.

I received this twice on this mailing list. Once from Lennart ("being a dick 
now") and now from you calling me a troll.

I will make an effort to reply to your mail and then most likely unsubscribe, 
cause for me I feel like being in an hostile environment.

Am Dienstag, 21. Oktober 2014, 22:14:48 schrieb Colin Guthrie:
> Martin Steigerwald wrote on 21/10/14 10:25:
> > Am Mittwoch, 8. Oktober 2014, 10:54:00 schrieb Lennart Poettering:
> >> On Tue, 07.10.14 23:40, Uoti Urpala (uoti.urpala at pp1.inet.fi) wrote:
> >>> On Tue, 2014-10-07 at 14:15 -0400, Rob Owens wrote:
> >>>> My question really isn't "why are the Debian dependencies the way they
> >>>> are".  I understand that.  I was trying to highlight the strange
> >>>> situation of a desktop application requiring a particular init system.
> >>>> I *think* this is a result of the init portion of systemd being bundled
> >>>> together with the logind portion of systemd.  To me (unfamiliar with
> >>>> the systemd code) those two functions seem distinct enough to merit
> >>>> being separate.  Debian can't easily separate what the systemd devs
> >>>> have developed as a single binary, so we end up with these strange
> >>>> dependency chains.>
> >>> 
> >>> "Single binary" is false of course. Logind is developed as a separate
> >>> program, which is why systemd-shim is possible at all.
> >>> 
> >>> AFAIK the actual relevant dependencies go as follows: First, there's a
> >>> good reason why logind requires cgroup functionality. And there's a good
> >>> reason why cgroup functionality is best implemented together with init
> >>> (see
> >>> http://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/
> >>> for more info). So it's not quite directly "logind has to depend on
> >>> systemd as init", but "logind has to depend on the system having cgroup
> >>> support, and there's no equally good cgroup support available for inits
> >>> other than systemd". It is possible to provide the relevant cgroup
> >>> interfaces in or on top of another init, as the systemd-sysv + cgmanager
> >>> combination attempts to do. But it is not trivial to do, as bugs and
> >>> implementation delays with that combo have shown, and it's quite likely
> >>> that there will be more problems in the future. It's not a particularly
> >>> good idea to use the less-tested and less-reliable systemd-shim instead
> >>> of the more reliable systemd. Thus the overall result is that yes, it
> >>> does make sense to switch machines to systemd when you add certain
> >>> functionality, even if that functionality does not appear to be directly
> >>> tied to the init system at first glance.
> >> 
> >> Also note that the systemd concepts logind makes use of are also
> >> exported in its own API. The "scopes" and "slices" that logind uses
> >> are exposed on its bus objects, and they are used by tools like
> >> "loginctl" to do their work.
> >> 
> >> The "scopes" and "slices" concept does not exist elsewhere, and
> >> there's nothing comparable around, so even if we wanted we couldn't
> >> make logind work on anything else.
> > 
> > Then why in the first hand are the "scopes" and "slices" concepts within
> > systemd *tightly* coupled when it is functionality that makes sense to be
> > utilitizes in a component that provides a different functionality.
> > 
> > I wonder what functionality systemd provides right now in one more or less
> > tightly coupled package:
> > 
> > - early boot process
> 
> PID 1
> 
> > - managing kernel drivers and device files (via coupled udev)
> 
> Not PID 1
> 
> > - resource control for process groups
> 
> PID 1.
> 
> > - logging
> 
> Not PID 1
> 
> > - core dumping
> 
> Not PID 1
> 
> > - network while I think a distro-unified way to approaching network that
> > replaces the current distro specific ways like ifupdown and so on, why
> > does it have to be coupled with systemd?
> 
> Not PID 1
> 
> > - cron, although that at least in Debian is separate as systemd-cron
> 
> Partly PID 1
> 
> > Thats rather much.
> > 
> > Really rather much.
> > 
> > Way more than traditonal sysvinit covered.
> 
> This is because traditional sysvinit covered PID 1 and performing it's
> job (if you consider e.g. killall5 and friends).
> 
> You seem to be conflating systemd as PID 1 and systemd as a project.
> These are two completely different things.

No. I am concerned about both. The functionality that is stuffed inside PID 1 
which is more than 1,3 MiB and also sports user session functionality. And 
what is coupled inside on project, more or less tight.

> There are many related things here and a lot of the sub components
> obviously use a lot of the same functionality and utility functions.
> This is achieved via a shared library with a private API.
> 
> The only reason that all these separate parts are developed as part of
> the systemd project is that it's *easier*. It's just the easiest way to
> develop.
> 
> An alternative would be to make the utility functions API stable and
> export the shared library publicly and give API guarantees, but that
> puts a lot of pressure and it's a difficult thing to provide and it has
> long term maintenance overhead. This is something that *costs*. It costs
> in time/man hours and therefore it carries real overheads. Doing this
> for the convenience of splitting things out is simply not worth it -
> especially so as the main people working on these things are the same
> people.

I don´t think it would be just for the convenience of splitting things. It 
would also be to address the concerns I summarized and that have been made 
elsewhere. You may heard these concerns often, you may not agree to them. Yet, 
if you say later that some of these concerns were made 5 years ago already… If 
those concerns are still there… either… its due to Debian adopting systemd 
now, but its not only Debian users and devels here giving that concerns… or it 
is due to that the once voicing the concerns have the impression that systemd 
upstream did not address them.

> It also increases the test matrix. If logind v300 has to work with
> libsystemd v300 and all future versions then the testing side of things
> increases in complexity *massively*. Again this causes problems that
> translate to time and effort of developers that could better be
> allocated to building a better overall set of building blocks.

I do think that the easiest way to do something is not necessarily the best.

It took KDE developers several years to split out KDE 4 into modules. Yet they 
did it. And from what I read on their mailinglists so far, usually their 
discussions are in a much more friendly tone.

Now systemd is not KDE, granted… but I think thats not the only example where 
developers took quite some extra effort in order to split things out.

> So overall, keeping things developed as one project is a technical
> decision that has real benefits both for the developers doing the work
> and the downstreams consuming it. Changing the approach would be 100%
> political and, quite frankly, the systemd developers have very little
> time for such games. They just want to get on and build better systems.

It would be 100% political if the concerns where 100% political. But in my 
oppinion they are not.

> > What of this actually *needs* to be coupled with systemd? What of this
> > needs to be coupled tightly to Linux kernel?
> 
> If people want to provide implementations of such components outside of
> systemd they would be more than welcome to do so.
> 
> As systemd provides a lot of the functionality and shared API functions
> that are convenient to use (i.e. the base building blocks) and as people
> who are working on the systemd project are doing the actual work, and as
> these are seen as core building blocks of a modern OS, then it makes
> sense to do this under the systemd umbrella.
> 
> There is no conspiracy here. It's just convenient. If other people want
> to do this work, great.

Again, again and again:

I never assumed bad intentions.

Well… as it doesn´t seem to sink in:

I never assumed a conspiracy.

I never assumed bad intentions.

Do you need it again?

Ok, here is:

I never assumed a conspiracy.

I never assumed bad intentions.


I see you received what I wrote as such and I am puzzled, cause I carefully 
took care to make that perfectly clear in all what I wrote so far.

It seems I still didn´t bring this point accross.

This may be due to history and Lennart and others here, systemd upstream 
having been called names and worse.

Yet *I* didn´t.
 
> > systemd is driving a road where its all of this functionality by default
> > is
> > only available for Linux and where upstream said they just won´t accept
> > patches – is that still true? – for portability.
> 
> systemd developer hs have stated this clearly several times over, but to
> say it again, it's not that systemd *won't* accept patches for
> portability, it's just that there will not be a codebase littered with
> #ifdefs and optional codepaths that are not well tested by the core devs.
> 
> systemd makes use of APIs that are currently *only* available on Linux.
> If other kernels implement the same APIs, then systemd will work with
> them just fine. If they implement equivilent functionality under
> different APIs, then it's unlikely the systemd developers would support
> this upstream directly as this would result in these #ifdefs and
> untestable paths (by the core developers) that are very unattractive for
> the upstream project.
> 
> But you know what? This doesn't matter! Those developers who are
> interested in those other kernels will just maintain their own fork of
> systemd (and by form I mean a true fork - one which git facilitates well
> where upstream changes can be merged periodically) where the linux calls
> are split out into $whatever calls.
> 
> This separate project can be tested by those developers and they can act
> as the first port of call for bugs in their version but collaborate with
> the original systemd for issues that are present in their upstream. In
> many cases the developers of this fork would have upstream commit access
> to. Such a world is perfectly feasible and it allows systemd to keep a
> clean codebase and the downstream too. This is one of the cases where
> committing everything upstream is not beneficial to either systemd or
> the fork. Both are more individually readable and maintainable by their
> relative maintainers. Everyone wins.

Ok, how would these forks address compatibility issues with upstream systemd? 
Upstream systemd has a very high development speed. Which you may view as 
good. And heck, yes, it has its advantages I agree. But to me it also seems 
that this speed partly come due to what you wrote above as the easy way of 
developing things. And that easy way to develop things, I argue now, makes it 
more difficult for people wanting to port to different platforms, people only 
wanting a subset of systemd and people who want to adapt systemd.

It is your free choice to handle things like this.

I just again and again also see the pattern of "This is not our business" 
where it was systemd creating friction for others. Heck, I even had this in 
some of my own bug reports regarding systemd in Debian where I was about to 
help systemd integration with Debian by pointing out issues.

I saw this here, I saw this with Kay Sievers and Linus reaction to him, I saw 
this on this mailing list with patches people send in.

I could put examples to it, but right now as I am feeling to be in an hostile 
environment and don´t want to stay up much longer this night, I won´t. For 
now.

> > systemd provides more and
> > more functionality that desktops like to use, that other tools like to
> > use.
> > 
> > When Microsoft back then did something like this it was called "Embrace,
> > Extend and Extinguish"¹…
> > 
> > a) Embrace: systemd provides functionality that is mostly compatible or
> > similar with what sysvinit and other components provided
> > 
> > b) Extend: systemd extends on that functionality and makes it more and
> > more
> > difficult for desktops and apps to ignore that kind of functionality
> > offers
> > 
> > c) Extinguish: The scope of systemd becomes so broad, the functionality it
> > offers in a – admittedly – convenient package to often needed and used,
> > that other ways of providing the functionality are extinguished.
> > 
> > Really… it matches quite closely.
> 
> Oh come on! This is just an attack and FUD. You make repeated claims of
> coming in good faith etc. and seem to dismiss any technical defence
> being made with vague references and then you bring out a aggressive and
> argumentative statement like the above.

That is the impression you get.

I think I replied to technical arguments as well.

But I also see more social and how do I develop things, how do I treat 
portability people, how do I treat long year sysadmins, how do I treat bug 
reports and so on… I see much of the issue in this.

Sure, doing it like this initially makes it easier for you.

But it also creates the outcome you perceive.

It triggers and summons the polarity it triggers and summons.

It is your choice.


> Again, you're totally confusing systemd as a project and systemd as a
> PID 1. You need to accept and understand that distinction before anyone
> here will take you even remotely seriously.

Okay, again my obversation:

1) Embrace: Systemd implements functionality that is available elsewhere. As 
PID 1 – service handling – and as project – DNS, DHCP, logging, handling core 
dumps… and others.

2) Extend: Systemd extends on this functionality. Which even makes sense. 
systemctl status is wonderful. But it still does. Again I see this for PID 1 
and for project.

3) Extinguish: More and more on the first sight unrelated projects, apps and so 
on depend on functionality that comes to distros as one entity: Systemd. More 
and more it becomes difficult to run a fully fledged LInux desktop with an 
alternative init system. More and more alternative init system developers need 
to play catchup game by implementing stuff systemd implements, but which was 
never fed into some standardization or stable API.

Again, again, and again: I see no bad intention. I see no conspiracy.

This is just what I *observe* and partly how I see similarities to Embrace, 
Extend and Extinguish.

Heck, I am not even sure Microsoft had this strategy from the beginning.

I am pretty sure you didn´t have this strategy. I am pretty sure Lennart did 
not have it. I am even pretty sure you or Lennart has it now.

But I see outcomes of the way how you currently handle things. Not *intended* 
outcomes, but still outcomes.

And I stand by it by pointing this out.

If you can´t handle this… fine… don´t. But thats how I see it.
 
> > I don´t think this was your plan. But… when looking at the currently
> > visible outcome this is quite exactly what is happening here.
> 
> No. Looking at the visible outcome of this to me is that huge progress
> is being made in documented APIs that solve real world projects backed
> up by solid, supported and maintained implementations. To me this is a
> great thing.

I am fine with this.

> You are reading into it what you want based on your
> preconceptions and then building up a case based on that. This is really
> a case of apophenia, but people just don't see that because they are so
> blinkered by their prejudices.

I am not fine with it. You don´t know what I do. You just interpret it.

As do I.

Its the only access I have to this world. I interpret what I see.

No what does make your interpretation more valid, than mine? What?

> > So if you can follow this just a tiny bit: Do you start to understand why
> > systemd triggers the uproar, polarity and split so obvious that one needs
> > to have hands before both eyes for not seeing it?
> 
> What do you expect? Ultimately, haters gonna hate. There are a group of
> people who are predisposed to hating change. systemd replaces a lot of
> core functionality. If that was rolled out *without* causing a group of
> people to complain I would have been surprised.

I seen feedback of haters. Yes.

But not almost all of the feedback I saw in debian-user, on debian-devel is 
that of haters or trolls.

Calling someone a hater or a troll is still calling names to me.

Its accusing and attacking persons to me.

What I tried to achieve is to describe and interpret what I see regarding the 
state of systemd as I understand it now, and granted my understanding may not 
be complete, sure, and also describe and interpret behavior I have seen. And 
also summarize some of this from the feedback I read elsewhere.

What I didn´t try to achieve was attacking persons.

Yet, I interpret your reaction to me as if I attacked you.

So I am obviously not producing the outcome I wanted to produce. And thats one 
reason why I think I will stop doing what I am doing after this mail and 
unsubscribe from this list for a while.

Actually I think I made my point. I tried to channel some of the dire concerns 
and uproar and polarity and split tendencies upstream.

I see this happening to my beloved distribution Debian and I am not happy 
about it. The systemd debates and polarity within Debian I consider as being 
harmful.

And it was my intention to address some of this upstream in order to discuss 
what can be done to first *understand* why it triggers this polarity and what 
can be done to address this.

Its causing harm.

Very much so. Right now. As we speak. And I am concerned about it.

I am confident that Debian as a project will manage. But I see the cost it has 
to the project. And I am very deeply concerned about it.

> Of course this criticism is listened to and often actions are taken
> because of it, but what do you expect the outcome to be? Do you expect
> all the repos to be split up? Stable APIs exported and supported? What
> outcome do you actually *want* here? You seem to be doing lots of
> complaining, but very little in the actual suggesting what could be done
> different that has not already been addressed technically. You may
> disagree about that end decision but that's just the way it is
> sometimes? The people doing the work ultimately get to make the decisions.

Yeah, thats the do-ocracy aspect of things. Still if what I do again and again 
and again triggers much of polarity and resistance, I´d ask myself whats going 
on there. Which brings me back to the point why I started this thread here.
 
> > So as I see it: there are *real* concerns. And if systemd is meant to be a
> > tool for users and admins, I *highly* and *strongly* recommend that you as
> > systemd closely look at these concerns. Unless you are willing to trigger
> > the polarity, the split and the uproar that it creates.
> > 
> > In KDE more and more voices come up that developers need to talk to their
> > users. Sure… its a do-ocracy. But still… if you want to become really
> > broadly accepted and successful with something you offer… its important
> > to listen for feedback. And in my oppinion you are in a good opportunity
> > here, cause I think there is no lack on feedback on systemd.
> 
> You also seem to think that this feedback is somehow special or new.
> Such feedback has been coming in for years and much of it has already
> been listened to and acted on or decided as something not to be
> supported or irrelevant. All of your posts so far seem to be worded like
> you think this process has not happened at all. There is a wealth of
> archive information, blog posts, conference talks and such like out
> there to document this process. You need to be proactive to consume it
> all, and you cannot expect to be spoon fed it all.

Well, okay. I don´t follow this list for that long already. I read here and 
there about systemd. Blog posts from Lennart but also critical feedback. Yet… 
as Debian was not affected, it was kinda remote to me. So… I think its 
understandable I don´t know five years of history.

I voice concerns I see now.

And as these concerns are now this tells to me that those who have the 
concerns either are not up to date that they have been address or otherwise 
don´t have the impression that it they got addresses. In some time concerns 
may have been addressed but information about it didn´t get throw. But in 
other cases it may just mean you deemed the concerns as being "irrelevant".

And this, in my eyes, it not quite a respectful way to deal with concerns.


> Honestly, I've been following this project since it's very inception and
> I from what I see your opinion of it has been formed around notions that
> are pretty much entirely fictional or based on second hand
> information/FUD. The accusations you make are often wildly technically
> inaccurate and you seem to offer no credit to the devlopers with regards
> to their collaboration.

This is your impression. I, as should be obvious right now, see this 
differently.

I see openness and technical discussion here, but I also see "not our 
business, go away" kind of reactions, in bug reports as someone pointed out in 
this thread already, but also here on this list.

Actually what you write above follows a similar pattern: Its the easiest way 
to develop things. Okay, it may is. For you. But did you consider the cost it 
creates for others?


> I've been doing the rounds on mailing lists, distros and conferences for
> years. systemd is one of the most open projects with regards to doing
> the conference circuit and talking to people - all over the world and
> with interested parties both commercial and community run. There is no
> end of opportunity to speak with the systemd developers - often in
> person - and discuss the issues. The regurgitation of the same old stuff
> on mailing lists today no longer warrants further discussion for the
> most part. Doing so just soaks up developer time (and it takes a *lot*
> of time to reply to these things - this email alone has likely taken me
> 20mins+ - I'd much rather developers give such "feedback" a brief read
> over for any salient points and then ignore it and get on with designing
> and writing code and making a difference.

It is your free choice to do with your time what you want to do. I am in no 
way forcing you to respond to my mail.

It seems to me you felt some urge to respond, as I felt urge to respond to 
you… thats okay. But thats something you feel or think, or I feel or think. So 
I give all responsibility on how you spend your time back to you.

I am not responsible for it.

As you are not responsible for that I read your mail so late and then not 
being able to sleep and just put it away till tomorrow.
 
> Sorry, but I just do not see things they way you do and your repeated
> regurgitation of factually incorrect statements is doing nothing but
> making you appear to be someone to not waste time replying to further.
> This is not ignorance or not listening to feedback or anything of the
> like... it's ultimately, I'm sorry to say, just not feeding the
> trolls[1] - or, if you prefer a more sanguine version, not replying is
> just agreeing to disagree.
> 
> Col
> 
> 1. for the avoidance of doubt, I don't mean anything nasty by this - I
> just mean replying just continues a pseudo "discussion" with someone who
> is not in any way listening to your arguments or accepting your
> technical opinion - it's just a never ending discussion that leads
> nowhere - it has to be starved to die off and cannot be fed.

That I received as a personal accusation.

I work with Linux for more than 10 years. I do system admin work, I give Linux 
trainings and and and…

Even if you try to do it politely. For me I received it as "You are a troll 
anyway."


Now I agree to disagree.

>From the thread so far I understand way better why people are concerned, why 
systemd as a project of developers, as a umbrella projects of building blocks 
as you call it and as PID 1 triggers the amount of polarity it does:

Or as Aaron Seigo carefully tried to put it:

Aaron Seigo, four paths, 7th of October 2014:
http://aseigo.blogspot.com/2014/10/four-paths.html

How the way Lennart and other developers here treat people as I perceive here 
and there also in how I feel being treated here has a contribution to the 
reaction and outcome they get. That doesn´t excuse name calling or worse. But 
it partly makes it understandable or explainable.

And as to the

"who is not in any way listening to your arguments or accepting your technical 
opinion"

I can give back. I even wrote to Lennart once I have the impression he doesn´t 
even relate to what I just wrote.

So I made my point. I made clear what my intentions were.

And I think there is nothing more to say for now.

I wish you success.

And thanks for putting the effort in the response you put into it.

Ciao,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7