State of Wayland protocol development

Wed Sep 30 02:35:33 PDT 2015

On Tue, Sep 29, 2015 at 09:14:26PM +0100, Daniel Stone wrote:
> Hi Jonas,
> 
> On 18 September 2015 at 08:00, Jonas Ådahl <jadahl at gmail.com> wrote:
> > Right now, the way to get a protocol officially declared stable is, more
> > or less, to implement it in weston, wait for a while maybe making
> > changes, and then when agreed upon moved to the wayland repository. While
> > being in weston, the corresponding .xml file may have been copied to some
> > other repository for being implemented elsewhere (as it should be). So
> > far I only know of one protocol that has gone through all the steps
> > (wl_subcompositor), the rest are still in weston or in the mailing list.
> 
> Even wl_scaler, which should be incredibly simple. :\
> 
> > A problem with the procedures we have is that it may be very hard to get
> > any protocols past the first step (posting to the mailing list), making
> > experimental adaptation harder, because the lack of versioning and the
> > fact that one would need to make a guess at what mailing list post is the
> > correct one at the moment. The reason for this I would say is unclear
> > requirements and expectations on what an unstable/experimental protocol
> > is, as well as unclear responsibilities on what group of people are
> > considered gate keepers of protocols.
> 
> Indeed, the mailing-list issue is a big one. Sometimes even I have a
> hard time tracking which of Pekka's branches should be current for a
> given WIP spec, which is a bit of a worry. (No slight on Pekka here,
> obviously, just our own lack of tracking.)
> 
> > In some cases, it may be more or less harmless to take a pending
> > experimental protocol and release it as an experimental feature. We
> > (GNOME) did this with pointer gestures, adding the latest version to
> > mutter and GTK+ in order to provide it as an experimental feature. The
> > problematic thing being that since, even though it is a generic protocol,
> > it does not have an "official" protocol version released anywhere even
> > though it is in use in a software release. While I believe it was pretty
> > harmless this time, I don't think this is a generally good way to do
> > things. For private protocols, within a private namespace, it is of
> > course the proper way to do it, but otherwise less so.
> 
> Actually, I think that is quite a good way to iterate it.

I disagree. I believe it makes it difficult for interoperability since
its unclear what versions of a protocol is "official" (as that version
number corresponds to that actual version). For example the one we have
released a version of (_wl_pointer_gestures) already has several versions
with the same version number in the mailing list, but only one in the
GNOME release. If we replace the mailing list with phabricator or
bugzilla, the issue is still the same; it should be considered a WIP
until its accepted.

> 
> > We could introduce a wayland-protocols that contains future stable
> > protocols as well as staging / experimental protocols, while only
> > installing stable ones. Weston, mutter, GTK+, kwin, etc would depend on
> > it, but for experimental protocols the .xml files would still need to be
> > copied, because otherwise one would have to depend on a specific release
> > since experimental protocols are not necessarily backward compatibility.
> > This would allow us to release protocols without having to wait for a
> > wayland/weston release cycle, and we wouldn't need to wait for an actual
> > implementation in weston to get a central point to keep track of
> > protocols in development. It could just as well be split into the wayland
> > or weston repositories just as it is now if the intention is to have the
> > same release schedule.
> >
> > The main issue, I believe, is that we lack defined procedure and agreed
> > upon requirements for what may actually be placed in such a repository. I
> > don't think it makes sense to have a sandbox protocol repository for
> > things like this; I think we need some system similar to what we have
> > with required review and discussion, and initial iterations and releases,
> > but I believe the pace of letting experimental protocols in and be
> > tracked not only on a mailing list, with proper versioning etc, is too
> > slow right now, and somehow we need to fix that.
> 
> Actually, I think a sandbox protocol repository could be a good one,
> particularly if we used git submodules to track it; someone did
> suggest this, but I can't find it now. The only downside to that is
> that it can tie versions together: you'd need to track zlinux_dmabuf
> and _wl_pointer_gestures together, which can be pretty onerous. Maybe
> the best approach would be to version the files: e.g. you'd have
> linux-dmabuf-v3.xml, linux-dmabuf-v4.xml, pointer-gestures-v1.xml,
> etc. That way we could have a single central repository for everyone,
> assuming they could be persuaded to use submodules.

The sandbox model has the same issue as iterating on the mailing list -
many versions with the same version number - or simply just too many
versions.

I do though think that the some-protocol-v3.xml together with a
repository with actual releases where the content has gone through some
kind of peer review makes sense though.

An alternative to a sandbox model could just as well be a wiki page with
a list of links to git branches. Then we don't even need to create fdo
accounts to everyone who wants to play in the sandbox. And when a
sandboxed protocol is ready for experimental adoption by more than one
project, it would go through the peer review and end up being merged to
our wayland-protocols repository.

> 
> > Do we need more people who should be considered gate keepers? Right now,
> > as far as I understand it, Pekka and Daniel are the ones who currently
> > are the de-facto gate keepers of actual protocol changes / additions, but
> > given how things look, should we consider defining some requirements for
> > releasing experimental / staging protocol versions, such as R-B's by at
> > least N out of a set of persons, and expanding this set of persons to a
> > few more? (Note that I'm not blaming anyone for taking time etc, I
> > completely understand the lack of time available to spend on reviewing
> > protocols and protocol implementations). For declaring a protocol stable
> > there should of course be stricter requirements than for experimental
> > ones, but also here we need to reconsider how we should do things
> > differently regarding changes/additions to already existing interfaces.
> 
> No, that's fine, I totally agree. I think there are a few issues here ...
> 
> Firstly, as you alluded to with finding the correct mailing list post,
> it's very difficult to track the current status of anything. I've been
> experimenting with Phabricator a bit (more in the Patchwork thread
> later), and I'd suggest that having a Phabricator task as a central
> place to track status - bearing in mind that you can also hang code
> review off that - would be a good start. If nothing else, it'd make
> clear the number of extensions which are stalled; thought to be
> largely complete, awaiting negative feedback which will never arrive
> to be marked stable.
> 
> Secondly, release scheduling can be quite ad-hoc, and we don't do a
> great job of tracking goals/targets for each release. That makes it
> pretty easy for things to slip a couple of releases, and potentially
> off the radar altogether if someone loses the time to work on Wayland
> (anyone remember the new test framework?). This is doubly true whilst
> we all have different time constraints: mine by various Collabora
> projects, others by GNOME/Fedora release cycles, others by Tizen
> products, etc. Without explicit goals and tracking, it's really
> difficult for us to co-ordinate some work.

I think it could improved some by decoupling "wayland-protocols" /
"wayland-unstable" releases from weston/wayland, allowing more liberal
iteration of protocols. It still has the issue you mentioned by coupling
of different experimental protocols, but it'd be less of an issue than a
sanbox approach IMHO.

We could also skip per-repository releases and just tag releases per
protocol, and rely on using it as a submodule or copying the xml files.

> 
> Thirdly, we definitely have given the impression of funnelling too
> much work towards certain people, and the previous two issues really
> exacerbate this. Using some particular toolkit-centric protocols as an
> example, neither Pekka nor I are always qualified to review them, yet
> they end up blocking on us for review. Even if they should ultimately
> have a quick pass through, being able to track the protocol status (is
> it being thrown out as an RFC? have Mutter/GTK+ already implemented
> and shipped it for two releases?) will make it a lot easier to work
> out what the committer group needs to put our time towards, and in
> what context.
> 
> The protocol repository approach seems like a good one, but I think
> it's really the more procedural side of things - rather than which
> repository it lives in - is what's really choking protocol development
> up at the moment.

Yes, I agree, but a protocol repository would allow us to implement some
procedural changes by decoupling releases it from weston/wayland. And by
skipping the inofficial requirement of having an implementation in weston
in order to make a protocol "official" we can still have a central peer
reviewed repository with some set of requirements and releases without
the need to have the R-B:s on the weston implementation.

Jonas

> 
> > What are the opinions of others here regarding this? Do we need to make
> > some procedural changes and if so what kind? Or are things working good
> > enough as they are?
> 
> I'm definitely not going to be the first to argue that it doesn't need
> improvement! Thanks a lot for bringing this up.
> 
> Cheers,
> Daniel
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel