State of Wayland protocol development

[Jonas Ådahl]

On Fri, Sep 18, 2015 at 04:35:52PM +0300, Pekka Paalanen wrote:
> On Fri, 18 Sep 2015 15:00:19 +0800
> Jonas Ådahl <jadahl at gmail.com> wrote:
> 
> > Hi,
> > 
> > I'd like to start a discussion on the state of how development of Wayland
> > interfaces are done and how they should be done, though less about the
> > technical aspect of it.
> > 
> > Right now, the way to get a protocol officially declared stable is, more
> > or less, to implement it in weston, wait for a while maybe making
> > changes, and then when agreed upon moved to the wayland repository. While
> > being in weston, the corresponding .xml file may have been copied to some
> > other repository for being implemented elsewhere (as it should be). So
> > far I only know of one protocol that has gone through all the steps
> > (wl_subcompositor), the rest are still in weston or in the mailing list.
> > 
> > Private protocols (for example gtk_shell) go through their own procedures
> > and I don't intend cover such protocols here. I'm also don't intend to
> > cover the desktop shell protocol core development that has been brought
> > up before[0] because it has other requirements, such as the fact that
> > without a shell protocol, a client cannot really do anything. Future
> > generic extensions to a stable xdg_shell are relevant here though.
> > 
> > A problem with the procedures we have is that it may be very hard to get
> > any protocols past the first step (posting to the mailing list), making
> > experimental adaptation harder, because the lack of versioning and the
> > fact that one would need to make a guess at what mailing list post is the
> > correct one at the moment. The reason for this I would say is unclear
> > requirements and expectations on what an unstable/experimental protocol
> > is, as well as unclear responsibilities on what group of people are
> > considered gate keepers of protocols.
> 
> Right, so should we be merging protocol proposals on much lower
> barrier, while agreeing on how we are going to communicate breaks in
> those protocols when to hey are developed and their current status?

Something like that, yes.

> 
> Sounds like a good plan to me, with the condition that changes to
> Weston's relatively stable parts must still be as carefully reviewed as
> usual. This would still lift the burden of ensuring a protocol is good
> before merging it.

I agree, but so far we have more people who actively merge code patches
than people that actively merge protocol patches, so maybe that won't be
as much of an issue.

> 
> If the protocol implementation details are well separated, maybe even
> in a plugin, then we could just land it with mostly acks.
> 
> All this requires clearly denoting which parts are more stable and
> which are considered experimental within Weston, both protocol files
> and implementations. Directory structure? Protocol interface names?

Are there any protocols in weston that are "stable"? Which ones would
that be? If it really were stable, wouldn't they be moved to wayland to
make it official? Sure we can have different levels of "unstable",
moving xml files between directories or something, but not sure what we
win doing that.

> 
> 
> Would it make more sense to have a repository separate from Weston for
> the protocols? *shrug*

The most obvious benefit would be that we can break an experimental
protocol, release the new version, but not yet update weston. If we put
it in weston, and change it, that change would need to be done to the
weston implementation as well. The awkward thing would then be that we
will have different versions of the xml files in different places. We
could also copy the xml file within weston and have the-protocol-1.xml,
the-protocol-2.xml etc. I don't really like the idea of having the
protocol version in the file name though.

> 
> Should we allow merging protocols into Weston without an implementation
> in Weston? Well, if you need a single central repository, and you are
> usually expected to give a Weston implementation sooner or later, then
> why not.

Well that is something we'd have to decide upon. Should all staging
unstable protocols have a weston implementation to be considered
"official"? I don't know. We could try I guess.

> 
> Is using Weston repository for protocols the right place? Considering
> the files should not be installed, but they mostly expect to have Weston
> implementations, yes. Would this be too slow? Well, there are people
> with push rights around, and if we agree on very loose requirements on
> pushing experimental protocol patches, maybe it wouldn't be too slow.
> 
> Should we be using a different Weston branch for including all the
> experimental stuff while leaving master for "stable development"? Nah,
> I think that might become a mess.[*]

I don't think we should. Branches will bit-rot and it will be a pain to
have to rebase them over and over again. They will also be harder to
discover and test. The main goal really is to make it more obvious where
to find in-progress protocols, and I don't think hiding them in branches
helps that much.

> 
> 
> What do you do when you do a backward-incompatible change to a protocol?
> 
> - Rename the globals or the protocol file? This would make people think
>   twice about the change as it would incur lots of churn in the code.
>   The churn shouldn't be a problem for review, since we wouldn't review
>   experimental implementations carefully. But it's still churn.

The way I've done it and also the way we did with pointer gestures is to
prefix all of the interfaces with an underscore, with a note in the
global interface's description describing the limitations and lack of
backward compatibility as well as future intentions. See here[0] for
exact wording.

A protocol bump can be backward compatible "by accident" and it'd be up
to the protocol designer to decide whether she/he wants to state any
compatibility, but the main idea is that a client should only bind the
exact same version it implements, and a server should terminate a client
that tries to bind an incompatible version.

The name of the XML file and the name of the protocol only affects the
generated file names, so I don't think we need to change them, but
please correct me if I'm wrong here.

Going stable would mean removing the prefix (i.e. changing all the
symbols etc), resetting the version to 1, and installing the XML file.
Regular backward compatibility semantics would now apply.

I think we should consider doing this to on all our experimental
protocols in weston (either that, or move them to wayland). And we
should probably move out the private protocols to a separate directory
as well.

> 
> - Use the mechanism tried with xdg-shell? A request to make-or-fail a
>   specific experimental version. This seemed like a good idea, we just
>   have to document it clearly and make sure it's followed properly.
>   E.g. the make-or-fail request *must* always be opcode 0. We could
>   have helpers for that.

No. That mechanism doesn't work, because it is not possible to discover
what version is implemented. A client that supports both xdg_shell and
wl_shell have no way of detecting (except try - be killed - try again)
whether it can use xdg_shell or wl_shell. The above described approach
does not have this limitation.

> 
> > So, how can we improve these procedures?
> > 
> > We could introduce a wayland-protocols that contains future stable
> > protocols as well as staging / experimental protocols, while only
> > installing stable ones. Weston, mutter, GTK+, kwin, etc would depend on
> > it, but for experimental protocols the .xml files would still need to be
> > copied, because otherwise one would have to depend on a specific release
> > since experimental protocols are not necessarily backward compatibility.
> > This would allow us to release protocols without having to wait for a
> > wayland/weston release cycle, and we wouldn't need to wait for an actual
> > implementation in weston to get a central point to keep track of
> > protocols in development. It could just as well be split into the wayland
> > or weston repositories just as it is now if the intention is to have the
> > same release schedule.
> 
> Or if we don't want to copy files, maybe we could use a ./configure
> option to point a build to checkout of the experimental protocol
> repository? It does have it's drawbacks, though.

This would make it harder to implement a newer version of experimental
protocol 1, while not wanting to implement a newer version of
experimental protocol 2. Except if we apply method 'copy-on-write' on
protocol version bumps.

> 
> 
> Btw. I have been thinking that when one stabilizes a protocol extension
> and moves the .xml file into the Wayland repository, we should also
> always rename the global interfaces and the .xml file. This would be
> the last protocol break, that allows us to remove any make-or-fail
> requests while also ensuring no-one will accidentally use it with old
> code, and it won't clash with any copies of the experimental xml file.

I agree. This is something I've taken into account when writing and
reviewing new protocols (as described above). But, what would be the
point of renaming the XML file?

The other part I brought up was adding to stable protocols. Currently
being discussed are additions to wl_data_device_manager (DND actions),
wl_seat (release request), wl_pointer (axis sources), wl_touch
(touch "focus" changes). Who are in charge of allowing such changes?
Some are quite huge (DND actions) and without a consensus, while the
other are relatively small, and thus with a consensus easier to reach,
but who's responsibility is it to make the call?


Jonas

[0] https://git.gnome.org/browse/mutter/tree/src/wayland/protocol/pointer-gestures.xml#n11

> 
> 
> [*] Hmm, on second thought, would it really? That's what topic branches
> are for, right? What if we did aiming-for-core protocol extensions in
> Weston topic branches? That way once the day comes to stabilise that
> extension, we can easily see what the patch to implement it in Weston
> looks like - it's just a merge away.
> 
> 
> Thanks,
> pq