State of Wayland protocol development

[Bryce Harrington]

On Tue, Sep 29, 2015 at 05:34:41PM +0100, Daniel Stone wrote:
> Hi,
> Leaving the more mechanical issue of how we deal with extension
> development for the moment ...
>
> > I don't think we've really had a maintainer since krh. I tried to take
> > over reviewing and releasing things and got exhausted, then Bryce took
> > over managing releases and I've tried to concentrate on areas I know
> > but I have limited energy.
> 
> For my part, I don't think we really need an active maintainer in the
> sense that we had krh at the beginning, or that we had pq
> subsequently. If you look at the role Bryce has played in the last few
> months, it's very different, and I think for the better - though in
> need of some tweaks.
>
> It's probably most helpful to look at the context in which we had our
> maintainers, and the way Wayland development has ebbed and flowed. krh
> was building Wayland quite literally from scratch, at which point you
> really need a single person to ensure that your vision is in any way
> coherent; especially when a lot of the work really involved working on
> Mesa/EGL, the kernel, and the toolkits simultaneously. By the time
> Kristian stepped out, Wayland had got to the point where it's entirely
> recognisable from today. I'd characterise the period of pq's active
> single-person maintenance as building Weston: taking it from a
> plaything to something shippable, not just radically remoulding it
> internally, but also keeping a very close eye on its scope.
> 
> Now with Wayland and Weston being bedded down, we have Bryce
> essentially acting more as a release manager than maintainer - where
> it's less about imposing or creating a vision, and more about ensuring
> that independent streams of development flow smoothly. I think this is
> pretty appropriate for the position we're in now.

Even though pq would deny it, I consider if anyone is the maintainer
it's him.  :-)

But yeah, my approach with projects tends to be pretty hands off except
around release time.  I figure the codebase is mature, and so are the
developers.  Maybe I trust people too easy but I know most people are
more knowledgeable than me, and as long as everyone is collaborating
well if there do come to be problems they'll work themselves out,
hopefully without too much drama. :-)

>From my perspective, the area where our maintenance process is
bottlenecked isn't in landing commits.  (Although, certainly wouldn't
hurt to have another committer!)  Rather, it's in *review*.

This is perfectly illustrated if you please refer to our newly upgraded
patchwork:

  http://patchwork.freedesktop.org/project/wayland/list/

Notice the three nice new A / R / T columns; these are measurements of
per-patch Acked-By, Reviewed-By, and Tested-By, respectively.  Next
notice that the list is zeros all the way down.  Patches that get R-B's
and A-B's and T-B's get landed, and typically fairly swiftly.

One thing this list *doesn't* show is the count of non-R-B reviews on
patches, which would be interesting.  But I think you'll find in
clicking through on patches that too many have no review comments at
all, or only very light ones, so it's still a "lack of reviewer" type of
problem.  (In some cases there is a deep thread of comment on a patchset
but it's not clear what the next action is, so the patch languishes;
someone needs to step in with a strong opinion to help decide a path
forward... again still a lack of reviewer type of issue.)

I suppose there's a tendancy to equate review responsibility and
committing responsibility, and leave it to our evidently theoretical
maintainer.  In practice I think those of us with commit rights feel a
sense of duty to tackle review work, or maybe we do it because otherwise
there wouldn't be much to land!  But as already mentioned, we each have
our own comfort areas we focus on, and sometimes more complex pieces get
left in limbo.  Some how, we have got to expand our collection of
reviewers, so we have adequate expertise to bear on all these areas.

But with this new Patchwork functionality we have another tool to track
review status, and I'm hopeful with incremental process improvements
like this, we'll be able to tackle the queue more effectively.

> Looking at the bigger issues, we have:
>   - the perennial xdg-shell issue
>   - the perennial testing issue
>   - libweston
>   - bulking up compositor-drm, e.g. with atomic modesetting
>   - working better with EGL/EGLImage, e.g. for dmabuf import
>   - Vulkan
>   - stabilising and bedding down several extensions (pointer-lock,
> scaler, presentation-time, IVI, etc)
>   - outreach to help external users, e.g. toolkits and other environments
>   - a long tail of misc and other

That's a good list, thanks.

To some degree, better coordination (like turning these items into a
tangible roadmap) could bring us benefits, and I have some ideas there.
But ultimately I feel what we need to be looking into is how to
stimulate more people to help with doing reviewing.  With a stronger
review culture, collectively our project will be better, individual
reviewers will broaden their experience, and contributors will see their
patches landed faster.

In particular, in our backlog I've noticed a number of our patches are
blocked not because of technical deficiencies but rather by the need of
some higher level decision being needed.  "Is this appropriate for
Weston?"  "Is this how we want this functionality implemented?"  "Should
the protocol be defined this way, or that way?"  These are the types of
issues where having defined expert "deciders" would be extremely
helpful.  It would be nice if we had a better way for an area expert to
flag/tag that they conceptually approve a given change's design,
separate from technical approval of its implementation.
 
> I don't think these necessitate a single gatekeeping maintainer in the
> way that Kristian and Pekka were, or in the way that Keith was for
> xserver until a week or two ago. I think each of these need a
> designated maintainer/go-to person: someone we trust knows the issues,
> someone we trust to be considerate of their impact on Wayland/Weston
> themselves and other wider projects, and someone we know works well
> with all involved with it.

I know the kernel has had good results with the lieutenant model, and
x.org does similar but more at a module level.  Where pieces can be
split off as their own distinct entities (e.g. libinput) I think it'll
be natural to do this.  But apart from that, personally I'm a bit
skeptical that weston is actually *big* enough to warrant such a model.

Certainly, we do have some chunks of code where there is a clear
development lead.  At least informally, I know us maintainers already
will often defer to a specific expert's judgment with larger code
changes, and just check that the changes meet our overall development
and release objectives.  Perhaps this could be expanded on.

The obvious worry here is in making it too formalized or rigid; we could
find ourselves replacing a single central bottleneck with an array of
smaller and tighter bottlenecks.

> This seems to be true of most all
> components; I've certainly had my concerns with xdg-shell's
> development (primarily how it seemed to have stalled a while ago, and
> its merge into Weston probably being premature), but it seems like we
> have more effective maintenance for the future.

xdg-shell is kind of a unique situation, as it by definition has to be a
product of collaboration and consensus.  You, me, and pq worked on a
proposal for better addressing its needs a while back; maybe would be
worthwhile to dust those ideas off, especially in light of some of the
other WIP protocol process needs recently discussed.

> Being all independent streams, they require different skillsets and
> backgrounds: I'll happily talk to you about KMS, about EGL
> integration, about media and presentation/timing, but I'm singularly
> unqualified to give my opinion on xdg-shell toolkit integration
> issues, on the whole.
> 
> Taking a step back, I think we should be looking at:
>   - how do we set the parameters for what we think is acceptable or
> good to have? really just classic release planning - set our dates and
> our boundaries, and any goals/expectations we have for those releases;
> something we've not always been great at, but with some more planning,
> discussion and visibility on list, I think we can do a much better job
> of this

Agreed.  And like I mentioned maybe your list above could be turned into
a proper roadmap.

>   - how do we best help people contribute within those boundaries? if
> someone is doing work we've all agreed on in principle, and it's got
> sufficient review, to me this is really about empowering them to push
> through directly; dropping the expectation that people other than
> Bryce/Pekka/myself pushing is some kind of weird event, or only done
> for trivialities

I'm certainly not opposed to adding more committers, but what do you
think of my thought above to have them instead mark stuff they approve
of with 'Acked-by' (or something roughly similar)?  I think this would
make for a pretty simple and low-risk workflow for them, and would
reduce workload on committers, who could just scoop up the A-B stuff and
land it without needing to do in-depth reviewing (unless they want to).

>   - how do we best help newcomers contribute? we're not always
> fantastic at getting review out, though Bryce/Derek/et al in
> particular have been fantastic at trying to sweep up Patchwork, and
> this does seem to improve

My philosophy is that the best way to encourage newcomers is to get
them a swift review.  Usually their stuff is straightforward and
so is quick to review and land.  So I usually start from the top of the
list, LIFO.  I notice Derek tends to gravitate towards the end of the
list, so covers FIFO.  You and pq appear to larger / more esoteric
bits.  So we have things covered from several angles.

One thing I've been thinking might help stimulate new developers would
be to have a "janitor's list" of simpler tasks.  For example there's
probably a number of basic improvements that could be done to the sample
clients, that we wouldn't bother roadmapping since they're just sample
clients.

And having an established roadmap will probably help too, as it helps
identify features that are sort of "pre-approved" for contributors that
want something meatier to dig into.

In general though, the time-tested way for getting more developers
involved is to establish some freeform-ish sandboxy area; so that's like
plugins, mods, experimental/contribs repos, or otherwise making
provisions to enable folks to establish up quasi-official side-projects.

>   - how do we avoid compromising on quality? I've been hoping to
> really beef up our CI to help with this

The nice thing about encouraging experts to give Acked-By's instead of
expanding commit rights, is that it augments our existing processes and
so increases QC rather than setting up paths around it.

Apart from encouraging more review, next would be more test case
diversity.  I've tried to encourage folks to include test cases with
their contributions, but I'm finding it hard to do that without becoming
annoying.  :-)

>   - how do we deal with anything outside the bounds we have now? what
> happens when something challenges the comfortable scope above?
> 
> This last question is definitely the more difficult one, and certainly
> we've been guilty of arguing around each other whilst these larger
> questions flail a little bit. But I think if we can resolve that, we
> move away from a requirement to have someone who will in all
> statistical probability try to burn out, and make better use of
> everyone's time.

Well said, and good topic for further discussion.  But I think I've
exhausted my char limit for this email.  :-)

Again, thanks for raising these issues.  I'd love to see us make better
progress on the backlog, and get stuff reviewed and landed more
reliably.

Bryce