Where should project Weston go?

[Pekka Paalanen]

On Tue, 9 Dec 2014 11:36:42 -0800
Jason Ekstrand <jason at jlekstrand.net> wrote:

> Bryce,
> Thanks for your thoughts.  I've got a few of my own, but I decided to
> reply to your e-mail as it seemed the best branch-point for the
> actual discussion without replying to everything.

Yes, very nice, I think I'm starting to see a pattern here.

It was excellent of Samuele to dig up the previous discussions, I
didn't even remember them!


> On Mon, Dec 8, 2014 at 3:26 PM, Bryce Harrington
> <bryce at osg.samsung.com> wrote:
> 
> > On Mon, Dec 08, 2014 at 02:01:32PM +0200, Pekka Paalanen wrote:
> > > Dear Wayland community,
> > >
> > > I would like to start a discussion on what Weston really is, and
> > > where it should go, if only to confirm that our concensus still
> > > holds.
> >
> > Thanks for opening discussion on this.  I suspect a lot of us have
> > been pondering these questions for a while.
> >
> > > To reiterate the main points:
> > > - the reference implementation
> >
> > > - fast (performant?)
> > > - suitable for embedded and mobile (and now automotive?)
> >
> > Performance is really only relevant in context to a use case;
> > otherwise this can lead us to pre-maturely optimizing stuff that no
> > one cares about.
> >
> > So, maybe "fast for embedded/mobile/automotive".  Or maybe there's
> > an implied use case in the original statement (i.e. fast desktop)
> > that should be made explicit.
> >
> 
> I think that Weston's performance matters a lot in some respects and
> not much in others.  We don't want to overcomplicate things by
> prematurely optimizing or squeezing out every clock cycle we can.
> However, Weston does give us the opportunity to show off some of cool
> performance things the Wayland protocol allows us to do.  One example
> of this is the planes support in the drm backend.  Weston may be the
> only desktop compositor around today that take a single subsurface
> out of a window, put it in a plane, and flip it directly to the
> screen.  This is fantastic for performance and power usage and is one
> of the advantages of the Wayland protocol.  If people come to us
> complaining that "you can't do that in Wayland" just because GNOME or
> KWin aren't doing it, we can point to Weston and say, "See, yes you
> can, they just aren't".  Another fantastic example of this is the way
> the RPi backend uses planes.  So I think showing off performance
> things we can do thanks to Wayland is important, squeezing out clock
> cycles isn't.

Totally.


> > > - minimal
> >
> > :-)  When I see a FOSS project describe itself as "lightweight" I
> > read it as "We consider our lack of features to be a Feature!"

Guilty. ;-)

> > But I imagine 'minimal' is intended here in more of an engineering
> > sense, and interpret it myself to mean something like: Focuses on
> > principle features not superfluous stuff better handled by other
> > projects; doesn't overengineer algorithms to squeeze a few drops of
> > performance; feature selection by what fits nicely and makes sense,
> > not by marketing demands; favors short and concise function
> > implementations that are easy to maintain and understand, even if
> > it limits performance/portability/features a bit.

I was going to say it's a convenient excuse to reject eccentric or too
complicated features, but I think you spelled it out perfectly. :-)


> > > Weston is also a test bed for new protocols and features. We tend
> > > to land experimental protocols in Weston, try them out there
> > > (while in this stage, third party projects need to copy the XML
> > > files from Weston), and when they stabilize, we move protocols
> > > into the Wayland project and install the XML files from there for
> > > everyone to use.
> >
> > Judging from what I've seen on the mailing lists there certainly is
> > a strong demand pushing weston to fulfil this role as a laboratory
> > of experimentation.
> >
> > But Weston's development processes seem to be set up more for
> > product development than for freeform R&D.  Namely, we have one
> > single "true" tree with a gatekeeper to control what gets
> > committed, reviewers to catch quality issues, and a growing
> > testsuite.
> >
> > I'm not saying this is wrong, just that this is probably not ideal
> > for facilitating experimentation; folks may prefer to gravitate
> > towards other compositor development projects where they can turn
> > ideas and patches around more swiftly.  And that's not necessarily
> > a bad thing... but it's different than considering weston as a test
> > bed.

Yes. The gatekeeper's goal is to keep Weston in a shape where it is
relatively easy to work with. I see that as a prerequisite for the
"easily forkable" below.

Having your own git repo is too easy with Github and others.


> Perhaps what we want here is "easily forkable".  Not that we want
> people to be trying to "fork the project" but that we want to make it
> easy for someone to work on a feature branch in their own github repo
> and make things easy to merge back in or rebase on upstream.  What
> does an "easily forkable" project look like?  That's a good question.
> 
> First would be to encourage people to work in feature branches rather
> than have to have everything on the list and/or in Weston core.
> While I think most things may want to be upstream eventually, trying
> to do upstream everything as you go is hard when people are trying to
> use it as a test-bed and you're trying to do stable release at the
> same time.  We may even want to allow stuff to go upstream by merges
> rather then everything as a linear series on the ML.

Agreed.


> Another thing might be to make things structured a little better to
> try and structure things in a more compartmentalized model.  Right
> now, Weston has a lot of files that are thousands of lines and should
> probably be broken up a bit (I'm looking at you, compositor.c).
> Having things structured a little better may make it easier for
> people to rebase.  That said, restructuring is a lot of churn and
> makes it hard to rebase, so that has to be taken into account too.
> 
> 
> > > Perhaps the most striking omission in the above is that the
> > > desktop for PC is not mentioned. The desktop is in scope only
> > > through "the reference implementation". What does it mean?
> > >
> > > I believe it means, that Weston needs to implement all the
> > > important desktop-related protocol extensions, obviously starting
> > > from xdg_shell. This is the reference implementation. However,
> > > other desktop-oriented protocol extensions might not be that
> > > obvious. How would we judge them? I guess it has to be case by
> > > case, which makes it hard for any one person to do.
> > >
> > > On the other hand, it also means that Weston does not need to, and
> > > maybe even should not, implement many desktop features that do not
> > > include a protocol extension to be standardized across Wayland
> > > servers.
> > >
> > > Where to draw the line? How to keep Weston "minimal" and "fast",
> > > while we would likely want to make Weston more usable as a
> > > desktop server too?
> > >
> > > Should we add one more goal for Weston: a simple desktop
> > > environment? But how simple? Would it include, say, a tool to do
> > > permanent video mode changes? A real screen lock? A task list?
> > > Also see the note about "camp agnostic" below, which limits the
> > > realistic possibilities a lot.
> > >
> > > Before one says "Yes!", we need to think about maintainability.
> > > Can we write a test that would run during 'make check' and see
> > > that the feature keeps on working? I agree with the saying that
> > > if something is not tested, you can assume it's broken. It is
> > > also a reason why I am so thrilled about the test suite
> > > improvements.
> > >
> > > So far the desktop shell implementation in Weston has not been
> > > considered anything more than a reference implementation of
> > > wl_shell and xdg_shell. Proper, more user friendly DEs are
> > > external projects that build Weston plugins. A problem there is
> > > that if Weston has to have a feature that is not used from the
> > > Weston project itself, it may easily break. There is also the
> > > problem that the plugin ABI is not stable.
> >
> > I've noticed over the years, that I can say until I'm blue in the
> > face that "Wayland is just a protocol, Weston is just a reference
> > implementation, and you need to look at desktop environments to
> > provide Wayland compositors;" but people still keep asking me,
> > "Okay, but when can I ditch X and just use Wayland as my desktop?"

Oh yeah...

> > Or the corollary: "If the Wayland protocol was finished some years
> > ago, and weston is 'just' a reference implementation, why is there
> > still development effort on it?"  The implication being that it's
> > being called a reference implementation much the way X.org is
> > "just" a reference implementation of X11.

I guess I haven't been paying attention to Phoronix forums. ;-)

> > There is a public [misperception?  expectation?  belief?] that
> > Wayland is a software product that will one day arrive and
> > revolutionize their desktop/smartphone/helicopter/dishwasher
> > experience.  Or at least fix some <mumble> problems that exist with
> > X's architecture.
> >
> >
> > So I figure there's two options.  One would be to chase this public
> > sentiment and start handling Weston as a regular desktop server
> > product development effort.  Weston's focus would need to be
> > restrained, as per the other requirements, so the desktop use case
> > would be limited to the audience for
> > "minimal/lightweight" (nvidia-free?) DEs.  If this is successful,
> > the law of the slippery slope will inevitably lead us to
> > implementing more and more features until it fulfills Zawinski's
> > law.

Right, maybe not. But...


> > The other option is to get weston's head and heart aligned, and
> > strengthen our focus on being "just" a test bed for improving the
> > Wayland protocol.  Give our community's primary attention not to
> > Weston's users but Wayland's users - Enlightenment, GNOME, KDE, and
> > other DEs.  Survey and study their requirements and look for things
> > they need across the board.  And make it easier for them to bring
> > us their problems/ideas/needs.  Make it convenient for people to do
> > prototyping and experimentation in Weston first, and then port to
> > production compositors, rather than vice versa.  Instead of unit
> > testing weston, improve the validation testing of wayland, such
> > that the same testsuite would run against the weston compositor,
> > enlightenment compositor, et al to verify that the DEs have
> > properly implemented the protocol requirements.  Instead of
> > delivering a monolithic Weston package, turn the best bits of it
> > into specialized libraries that DEs can use too (if they wish).
> > Define a clear process for 3rd parties to contribute extensions,
> > libraries, and driver support changes.  Finally, work to change
> > public perception to stop the expectations on producing
> > Weston-the-desktop, by vocally championing DEs that are already
> > giving solid Wayland-based desktops.

Yes. I'm guilty of pretty much ignoring Wayland-fits. Maybe we should
look in that direction for testing?


> Yes please!  One thing that has always bothered me is the existance
> of a "weston community" of people trying to make Weston the DE
> happen.  I think that the Wayland world will need a place for the
> lightweight desktop such as icewm and i3 are for X and maybe Weston
> is the place to do that. However, I think that users and potential
> future developers should be encouraged to try and help one of the
> other major DE's get their Wayland implementations off the ground.
> The other thing the Wayland-specific community can do is, as Bryce
> mentioned, develop tools and test suites to help implementers.
> 
> There's one big caveat to what I just said that I think is worth
> mentioning.  Namely, that making libraries and things for DE's to use
> is something that needs to be guided by DE development itself.  You
> can't really start a helper project without someone who's familiar
> with big DE's and what they need.  That said, I'm sure libinput
> wouldn't mind a little help and maybe other library projects will
> spring up.

Very much. I believe finding such parts that could be turned into
cross-DE libraries are hard to come by, and cannot be invented - they
emerge from a common need, like libinput.

> The other area (that Bryce mentioned in a different area) is tools.
> Protocol dumpers are great.  Other things that allow for better
> introspection or make it easier to implement the protocol would be
> fantastic.
> 
> 
> >
> >
> > > Then what should we not put in Weston?
> > >
> > > So far Weston has been kept deliberately "camp" agnostic, which
> > > means that dependencies to GNOME, KDE, E or similar "camps" are
> > > not allowed. In practice, Weston should never even optionally
> > > depend on GTK+, Qt, EFL, or such. Weston does optionally depend
> > > on glib, but I suppose that is acceptable considering the
> > > features depending on it. It has been camp agnostic to not favour
> > > any particular existing DE camp, and to not alienate any other DE
> > > camp.
> > >
> > > I think walking that middle road has been good, and we should
> > > continue that.
> > >
> > > What do you think what Weston should be in the future?
> >
> > Because Weston has walked this middle road, it puts it in the
> > position of being able to serve as a standards body for the DE
> > community. Historically that's been a huge need but not always well
> > delivered on. freedesktop.org's mailman is thick with efforts that
> > tried to establish standard solutions to common needs, with
> > successes ranging from full to nil.  So, for the overall FOSS
> > community we're in a very valuable spot, that would be very hard to
> > establish from scratch.
> >
> > I'd love to see Weston become a proper desktop server, and think
> > personally it'd be fun to work on.  However, it could be counter
> > productive from a larger perspective.  Even if we refrain from
> > using Qt, gtk, efl, etc. the fact that we'd be making (and
> > presumably promoting) a usable alternative could be seen as
> > disrespectful of the DEs that have already done the implementation
> > work for adding Wayland support.

That's an interesting point of view.

> > Better might be to look for feature needs common to all the DEs that
> > could be candidate for addition to the Wayland protocol, or go into
> > a new or existing library that all DEs could share.  Weston's focus
> > would then be the R&D of those bits, and creating associated
> > documentation and validation testcases.
> >
> >
> > > Btw. I still would not claim to be *the* Wayland/Weston
> > > maintainer, even though lately I have practically been doing that
> > > as time permits (sponsored by Collabora). I would like to see
> > > more of capable people rising as reviewers and some through
> > > experience eventually to maintainership. I cannot do this alone
> > > forever like Kristian did.
> >
> 
> Kristian couldn't do it alone forever either.  Why do you think you're
> doing it. :-P  For a while, I was trying to keep somewhat on top of
> things, but I don't have much time for it these days.

Given the time period he did, it was practically forever. :-)


> I know from looking at patchwork lately how big of a workload this is
> > for one person.  I would be more than happy to lend a hand with
> > maintenance; I do maintenance on Cairo, am release manager for
> > Inkscape and used to maintain the X stack for Ubuntu, so have some
> > experience here.  Samsung has a strong interest in Wayland right
> > now, so the maintenance work would be a day job priority.
> 
> 
> If your managers at Samsung are ok with it, that would be fantastic!
> If nothing else, making sure things get reviewed.

Yes!

----->8------


So looking at this sub-thread, I think I see some common directions,
also taking into account the old discussions that Samuele dug up.

- Weston's core should be stable, practically production-quality.

- Implementing a DE is a non-goal.

- We should perhaps move to simpler implementations if possible, to
  promote being a reference implementation.

- Performance is essential when we leverage hardware resources in smart
  ways; not by optimizing code below algorithm level, or even at
  algorithm level.

- When writing tests in Weston, they should be testing things specific
  to Weston. Any compositor-agnostic features would preferrably be
  tested by Wayland-fits or such.

I'm not sure I expressed that well, but I have a feeling the following
great long term goals might be appropriate for the desktop side:

- again remember that the Weston DE is a non-goal

- instead, help external projects to develop a DE based on Weston:

- embrace the libweston architecture; maybe develop it even further

- gut shell.c: leave only a reference implementation of xdg_shell in
  Weston repository, remove wl_shell support

- external project(s) should take what desktop-shell is today, and
  develop it further towards a DE / a demo case (we need some place to
  have the bling)

- at some point, hopefully the different Weston xdg_shell
  implementations converge such that a sensible window management API
  falls out, creating a new type of sub-plugins: window managers

- a Weston-based external DE project could choose to replace the helper
  client, the window manager plugin, the shell protocol implementation,
  or everything, depending on how complicated things it wants to do.

I would imagine all this to happen over a long period of time. Maybe we
could start by getting the libweston series merged after 1.7.0.
Practically all of the mentioned items need to happen naturally, guided
mostly by what we merge into Weston, and preferably leading development
to follow this direction even before any patches are written.

Slowly, gently... what do you think?


My thoughts are starting to wonder, and I haven't yet read the other
sub-thread of this discussion, so I'll get back another day.


Thanks,
pq