State of Wayland protocol development

[Derek Foreman]

On 18/09/15 11:12 AM, Giulio Camuffo wrote:
> 2015-09-18 17:44 GMT+03:00 Jonas Ådahl <jadahl at gmail.com>:
>> On Fri, Sep 18, 2015 at 04:35:52PM +0300, Pekka Paalanen wrote:
>>> On Fri, 18 Sep 2015 15:00:19 +0800
>>> Jonas Ådahl <jadahl at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'd like to start a discussion on the state of how development of Wayland
>>>> interfaces are done and how they should be done, though less about the
>>>> technical aspect of it.
>>>>
>>>> Right now, the way to get a protocol officially declared stable is, more
>>>> or less, to implement it in weston, wait for a while maybe making
>>>> changes, and then when agreed upon moved to the wayland repository. While
>>>> being in weston, the corresponding .xml file may have been copied to some
>>>> other repository for being implemented elsewhere (as it should be). So
>>>> far I only know of one protocol that has gone through all the steps
>>>> (wl_subcompositor), the rest are still in weston or in the mailing list.
>>>>
>>>> Private protocols (for example gtk_shell) go through their own procedures
>>>> and I don't intend cover such protocols here. I'm also don't intend to
>>>> cover the desktop shell protocol core development that has been brought
>>>> up before[0] because it has other requirements, such as the fact that
>>>> without a shell protocol, a client cannot really do anything. Future
>>>> generic extensions to a stable xdg_shell are relevant here though.
>>>>
>>>> A problem with the procedures we have is that it may be very hard to get
>>>> any protocols past the first step (posting to the mailing list), making
>>>> experimental adaptation harder, because the lack of versioning and the
>>>> fact that one would need to make a guess at what mailing list post is the
>>>> correct one at the moment. The reason for this I would say is unclear
>>>> requirements and expectations on what an unstable/experimental protocol
>>>> is, as well as unclear responsibilities on what group of people are
>>>> considered gate keepers of protocols.
>>>
>>> Right, so should we be merging protocol proposals on much lower
>>> barrier, while agreeing on how we are going to communicate breaks in
>>> those protocols when to hey are developed and their current status?
>>
>> Something like that, yes.
>>
>>>
>>> Sounds like a good plan to me, with the condition that changes to
>>> Weston's relatively stable parts must still be as carefully reviewed as
>>> usual. This would still lift the burden of ensuring a protocol is good
>>> before merging it.
>>
>> I agree, but so far we have more people who actively merge code patches
>> than people that actively merge protocol patches, so maybe that won't be
>> as much of an issue.
>>
>>>
>>> If the protocol implementation details are well separated, maybe even
>>> in a plugin, then we could just land it with mostly acks.
>>>
>>> All this requires clearly denoting which parts are more stable and
>>> which are considered experimental within Weston, both protocol files
>>> and implementations. Directory structure? Protocol interface names?
>>
>> Are there any protocols in weston that are "stable"? Which ones would
>> that be? If it really were stable, wouldn't they be moved to wayland to
>> make it official? Sure we can have different levels of "unstable",
>> moving xml files between directories or something, but not sure what we
>> win doing that.
>>
>>>
>>>
>>> Would it make more sense to have a repository separate from Weston for
>>> the protocols? *shrug*
>>
>> The most obvious benefit would be that we can break an experimental
>> protocol, release the new version, but not yet update weston. If we put
>> it in weston, and change it, that change would need to be done to the
>> weston implementation as well. The awkward thing would then be that we
>> will have different versions of the xml files in different places. We
>> could also copy the xml file within weston and have the-protocol-1.xml,
>> the-protocol-2.xml etc. I don't really like the idea of having the
>> protocol version in the file name though.
>>
>>>
>>> Should we allow merging protocols into Weston without an implementation
>>> in Weston? Well, if you need a single central repository, and you are
>>> usually expected to give a Weston implementation sooner or later, then
>>> why not.
>>
>> Well that is something we'd have to decide upon. Should all staging
>> unstable protocols have a weston implementation to be considered
>> "official"? I don't know. We could try I guess.
>>
>>>
>>> Is using Weston repository for protocols the right place? Considering
>>> the files should not be installed, but they mostly expect to have Weston
>>> implementations, yes. Would this be too slow? Well, there are people
>>> with push rights around, and if we agree on very loose requirements on
>>> pushing experimental protocol patches, maybe it wouldn't be too slow.
>>>
>>> Should we be using a different Weston branch for including all the
>>> experimental stuff while leaving master for "stable development"? Nah,
>>> I think that might become a mess.[*]
>>
>> I don't think we should. Branches will bit-rot and it will be a pain to
>> have to rebase them over and over again. They will also be harder to
>> discover and test. The main goal really is to make it more obvious where
>> to find in-progress protocols, and I don't think hiding them in branches
>> helps that much.
>>
>>>
>>>
>>> What do you do when you do a backward-incompatible change to a protocol?
>>>
>>> - Rename the globals or the protocol file? This would make people think
>>>   twice about the change as it would incur lots of churn in the code.
>>>   The churn shouldn't be a problem for review, since we wouldn't review
>>>   experimental implementations carefully. But it's still churn.
>>
>> The way I've done it and also the way we did with pointer gestures is to
>> prefix all of the interfaces with an underscore, with a note in the
>> global interface's description describing the limitations and lack of
>> backward compatibility as well as future intentions. See here[0] for
>> exact wording.
>>
>> A protocol bump can be backward compatible "by accident" and it'd be up
>> to the protocol designer to decide whether she/he wants to state any
>> compatibility, but the main idea is that a client should only bind the
>> exact same version it implements, and a server should terminate a client
>> that tries to bind an incompatible version.
> 
> We could also add a "_N" suffix to the interface name, where N would
> be the version number. This way a client can easily check the version
> of the interface implemented and only bind the global name with the
> right version, and at the same time it allows to do backward
> compatible changes to the interface by increasing the normal interface
> version.
> Moreover, if a compositor feels like it, it can implement many
> different incompatible versions of the interface at the same time.
> When stable, the _N suffix would be removed.

This does have some appeal from EFL/Enlightenment's perspective.  We've
already been discussing keeping multiple old versions of xdg-shell.xml
around and trying to find a way to ensure that anything we supported in
a release continues to be supported... somehow.

Crazy maybe, I dunno.  But your proposal seems to makes it possible with
a reduction in effort...

>>
>> The name of the XML file and the name of the protocol only affects the
>> generated file names, so I don't think we need to change them, but
>> please correct me if I'm wrong here.
>>
>> Going stable would mean removing the prefix (i.e. changing all the
>> symbols etc), resetting the version to 1, and installing the XML file.
>> Regular backward compatibility semantics would now apply.
>>
>> I think we should consider doing this to on all our experimental
>> protocols in weston (either that, or move them to wayland). And we
>> should probably move out the private protocols to a separate directory
>> as well.
>>
>>>
>>> - Use the mechanism tried with xdg-shell? A request to make-or-fail a
>>>   specific experimental version. This seemed like a good idea, we just
>>>   have to document it clearly and make sure it's followed properly.
>>>   E.g. the make-or-fail request *must* always be opcode 0. We could
>>>   have helpers for that.
>>
>> No. That mechanism doesn't work, because it is not possible to discover
>> what version is implemented. A client that supports both xdg_shell and
>> wl_shell have no way of detecting (except try - be killed - try again)
>> whether it can use xdg_shell or wl_shell. The above described approach
>> does not have this limitation.
>>
>>>
>>>> So, how can we improve these procedures?
>>>>
>>>> We could introduce a wayland-protocols that contains future stable
>>>> protocols as well as staging / experimental protocols, while only
>>>> installing stable ones. Weston, mutter, GTK+, kwin, etc would depend on
>>>> it, but for experimental protocols the .xml files would still need to be
>>>> copied, because otherwise one would have to depend on a specific release
>>>> since experimental protocols are not necessarily backward compatibility.
>>>> This would allow us to release protocols without having to wait for a
>>>> wayland/weston release cycle, and we wouldn't need to wait for an actual
>>>> implementation in weston to get a central point to keep track of
>>>> protocols in development. It could just as well be split into the wayland
>>>> or weston repositories just as it is now if the intention is to have the
>>>> same release schedule.
>>>
>>> Or if we don't want to copy files, maybe we could use a ./configure
>>> option to point a build to checkout of the experimental protocol
>>> repository? It does have it's drawbacks, though.
>>
>> This would make it harder to implement a newer version of experimental
>> protocol 1, while not wanting to implement a newer version of
>> experimental protocol 2. Except if we apply method 'copy-on-write' on
>> protocol version bumps.
>>
>>>
>>>
>>> Btw. I have been thinking that when one stabilizes a protocol extension
>>> and moves the .xml file into the Wayland repository, we should also
>>> always rename the global interfaces and the .xml file. This would be
>>> the last protocol break, that allows us to remove any make-or-fail
>>> requests while also ensuring no-one will accidentally use it with old
>>> code, and it won't clash with any copies of the experimental xml file.
>>
>> I agree. This is something I've taken into account when writing and
>> reviewing new protocols (as described above). But, what would be the
>> point of renaming the XML file?
>>
>> The other part I brought up was adding to stable protocols. Currently
>> being discussed are additions to wl_data_device_manager (DND actions),
>> wl_seat (release request), wl_pointer (axis sources), wl_touch
>> (touch "focus" changes). Who are in charge of allowing such changes?
>> Some are quite huge (DND actions) and without a consensus, while the
>> other are relatively small, and thus with a consensus easier to reach,
>> but who's responsibility is it to make the call?
>>
>>
>> Jonas
>>
>> [0] https://git.gnome.org/browse/mutter/tree/src/wayland/protocol/pointer-gestures.xml#n11
>>
>>>
>>>
>>> [*] Hmm, on second thought, would it really? That's what topic branches
>>> are for, right? What if we did aiming-for-core protocol extensions in
>>> Weston topic branches? That way once the day comes to stabilise that
>>> extension, we can easily see what the patch to implement it in Weston
>>> looks like - it's just a merge away.
>>>
>>>
>>> Thanks,
>>> pq
>>
>>
>> _______________________________________________
>> wayland-devel mailing list
>> wayland-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
>