State of Wayland protocol development

[Daniel Stone]

Hi Jonas,

On 18 September 2015 at 08:00, Jonas Ådahl <jadahl at gmail.com> wrote:
> Right now, the way to get a protocol officially declared stable is, more
> or less, to implement it in weston, wait for a while maybe making
> changes, and then when agreed upon moved to the wayland repository. While
> being in weston, the corresponding .xml file may have been copied to some
> other repository for being implemented elsewhere (as it should be). So
> far I only know of one protocol that has gone through all the steps
> (wl_subcompositor), the rest are still in weston or in the mailing list.

Even wl_scaler, which should be incredibly simple. :\

> A problem with the procedures we have is that it may be very hard to get
> any protocols past the first step (posting to the mailing list), making
> experimental adaptation harder, because the lack of versioning and the
> fact that one would need to make a guess at what mailing list post is the
> correct one at the moment. The reason for this I would say is unclear
> requirements and expectations on what an unstable/experimental protocol
> is, as well as unclear responsibilities on what group of people are
> considered gate keepers of protocols.

Indeed, the mailing-list issue is a big one. Sometimes even I have a
hard time tracking which of Pekka's branches should be current for a
given WIP spec, which is a bit of a worry. (No slight on Pekka here,
obviously, just our own lack of tracking.)

> In some cases, it may be more or less harmless to take a pending
> experimental protocol and release it as an experimental feature. We
> (GNOME) did this with pointer gestures, adding the latest version to
> mutter and GTK+ in order to provide it as an experimental feature. The
> problematic thing being that since, even though it is a generic protocol,
> it does not have an "official" protocol version released anywhere even
> though it is in use in a software release. While I believe it was pretty
> harmless this time, I don't think this is a generally good way to do
> things. For private protocols, within a private namespace, it is of
> course the proper way to do it, but otherwise less so.

Actually, I think that is quite a good way to iterate it.

> We could introduce a wayland-protocols that contains future stable
> protocols as well as staging / experimental protocols, while only
> installing stable ones. Weston, mutter, GTK+, kwin, etc would depend on
> it, but for experimental protocols the .xml files would still need to be
> copied, because otherwise one would have to depend on a specific release
> since experimental protocols are not necessarily backward compatibility.
> This would allow us to release protocols without having to wait for a
> wayland/weston release cycle, and we wouldn't need to wait for an actual
> implementation in weston to get a central point to keep track of
> protocols in development. It could just as well be split into the wayland
> or weston repositories just as it is now if the intention is to have the
> same release schedule.
>
> The main issue, I believe, is that we lack defined procedure and agreed
> upon requirements for what may actually be placed in such a repository. I
> don't think it makes sense to have a sandbox protocol repository for
> things like this; I think we need some system similar to what we have
> with required review and discussion, and initial iterations and releases,
> but I believe the pace of letting experimental protocols in and be
> tracked not only on a mailing list, with proper versioning etc, is too
> slow right now, and somehow we need to fix that.

Actually, I think a sandbox protocol repository could be a good one,
particularly if we used git submodules to track it; someone did
suggest this, but I can't find it now. The only downside to that is
that it can tie versions together: you'd need to track zlinux_dmabuf
and _wl_pointer_gestures together, which can be pretty onerous. Maybe
the best approach would be to version the files: e.g. you'd have
linux-dmabuf-v3.xml, linux-dmabuf-v4.xml, pointer-gestures-v1.xml,
etc. That way we could have a single central repository for everyone,
assuming they could be persuaded to use submodules.

> Do we need more people who should be considered gate keepers? Right now,
> as far as I understand it, Pekka and Daniel are the ones who currently
> are the de-facto gate keepers of actual protocol changes / additions, but
> given how things look, should we consider defining some requirements for
> releasing experimental / staging protocol versions, such as R-B's by at
> least N out of a set of persons, and expanding this set of persons to a
> few more? (Note that I'm not blaming anyone for taking time etc, I
> completely understand the lack of time available to spend on reviewing
> protocols and protocol implementations). For declaring a protocol stable
> there should of course be stricter requirements than for experimental
> ones, but also here we need to reconsider how we should do things
> differently regarding changes/additions to already existing interfaces.

No, that's fine, I totally agree. I think there are a few issues here ...

Firstly, as you alluded to with finding the correct mailing list post,
it's very difficult to track the current status of anything. I've been
experimenting with Phabricator a bit (more in the Patchwork thread
later), and I'd suggest that having a Phabricator task as a central
place to track status - bearing in mind that you can also hang code
review off that - would be a good start. If nothing else, it'd make
clear the number of extensions which are stalled; thought to be
largely complete, awaiting negative feedback which will never arrive
to be marked stable.

Secondly, release scheduling can be quite ad-hoc, and we don't do a
great job of tracking goals/targets for each release. That makes it
pretty easy for things to slip a couple of releases, and potentially
off the radar altogether if someone loses the time to work on Wayland
(anyone remember the new test framework?). This is doubly true whilst
we all have different time constraints: mine by various Collabora
projects, others by GNOME/Fedora release cycles, others by Tizen
products, etc. Without explicit goals and tracking, it's really
difficult for us to co-ordinate some work.

Thirdly, we definitely have given the impression of funnelling too
much work towards certain people, and the previous two issues really
exacerbate this. Using some particular toolkit-centric protocols as an
example, neither Pekka nor I are always qualified to review them, yet
they end up blocking on us for review. Even if they should ultimately
have a quick pass through, being able to track the protocol status (is
it being thrown out as an RFC? have Mutter/GTK+ already implemented
and shipped it for two releases?) will make it a lot easier to work
out what the committer group needs to put our time towards, and in
what context.

The protocol repository approach seems like a good one, but I think
it's really the more procedural side of things - rather than which
repository it lives in - is what's really choking protocol development
up at the moment.

> What are the opinions of others here regarding this? Do we need to make
> some procedural changes and if so what kind? Or are things working good
> enough as they are?

I'm definitely not going to be the first to argue that it doesn't need
improvement! Thanks a lot for bringing this up.

Cheers,
Daniel