State of Wayland protocol development

[Carlos Garnacho]

Hi Pekka,

On Mon, Sep 21, 2015 at 2:28 PM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> On Fri, 18 Sep 2015 22:44:16 +0800
> Jonas Ådahl <jadahl at gmail.com> wrote:
>
>> On Fri, Sep 18, 2015 at 04:35:52PM +0300, Pekka Paalanen wrote:
>> > On Fri, 18 Sep 2015 15:00:19 +0800
>> > Jonas Ådahl <jadahl at gmail.com> wrote:
>> >
>> > > Hi,
>> > >
>> > > I'd like to start a discussion on the state of how development of Wayland
>> > > interfaces are done and how they should be done, though less about the
>> > > technical aspect of it.
>
>> > All this requires clearly denoting which parts are more stable and
>> > which are considered experimental within Weston, both protocol files
>> > and implementations. Directory structure? Protocol interface names?
>>
>> Are there any protocols in weston that are "stable"? Which ones would
>> that be? If it really were stable, wouldn't they be moved to wayland to
>> make it official? Sure we can have different levels of "unstable",
>> moving xml files between directories or something, but not sure what we
>> win doing that.
>
> Weston's internal protocol interfaces (e.g. screenshooter.xml) are
> essentially stable - more precisely, they should follow Weston's plugin
> ABI stability rules. This is something we have never talked about
> before, but I think it makes sense. Maynard is an external project
> using desktop-shell.xml and needs a plugin to work.
>
> Not only protocols, but Weston's code also is important to classify.
> Weston's core is used in many places and we'd like to keep the core
> stable (as in not too buggy). Things like desktop-shell/*.c or
> ivi-shell/*.c are more on the unstable side.
>
> All these have even now slightly different review requirements we
> haven't really talked about, so I think people do it by feeling, erring
> on the safe side.
>
>> > Should we be using a different Weston branch for including all the
>> > experimental stuff while leaving master for "stable development"? Nah,
>> > I think that might become a mess.[*]
>>
>> I don't think we should. Branches will bit-rot and it will be a pain to
>> have to rebase them over and over again. They will also be harder to
>> discover and test. The main goal really is to make it more obvious where
>> to find in-progress protocols, and I don't think hiding them in branches
>> helps that much.
>
> The idea was to periodically merge master into the topic-branch, not
> rebase. So a topic-branch would be easily at least as recent as master.
>
> I first thought one branch for everything experimental and rejected
> that idea, but topic-branches is an idea I haven't rejected yet.
>
> When someone wants to propose a topic branch to be merged, it is easy
> to generate the diff to master. Reviewing that would be hopefully
> easier than rebase/squash/resend cycle, and it wouldn't invalidate old
> testing.

IMO Topic branches still have a problem of discoverability (although
to a lesser extent than patches in the ML). Branches in the main
wayland/weston repos would seem the most "official" way, but can be
done only by the selected few. Others (amongst them me) have fdo
account, so the feature branches could at least be in a ~username repo
at git.freedesktop.org. For other less fortunate ones, there might not
be other option than github or others.

Let me suggest something similar to gstreamer bad plugins, having a
"wayland-unstable" repo that gathers all latest versions of non-core
protocols. Unfortunately these are nothing without an implementation,
so I'd suggest making this repo compile into a noinst library for
basic tests, and require compositors to copy xmls from there. The push
requirements could be more relaxed (eg. any patch s-b or r-b the
maintainers or the authors of the specific protocol gets in,
implementations somewhere other than weston suffice to get this
in,...), this is minor maintenance burden that might be even
delegated. Ideally but not mandatorily, updates to a protocol in that
repo should come together with weston patches that would get the
full-blown protocol+code review, but the unstable repo would always
contain the last instance of the protocol. And of course these could
become "core", at which point they should probably be removed from the
repo and bumped (versioning, removing prefixed '_', whatnot).

IMHO the advantages are:

- All unstable protocol pieces are in a central place
- The pre-stable history is preserved somewhere, instead of scattered
across the compositors/toolkits that dare implement that.
- it allows developers of other compositors to play early with the
protocol and chime in ASAP, maybe they could even lead to stabilized
protocol changes before the weston patches get proper review, and
higher quality patches left for you to review in the end.

>
>> > What do you do when you do a backward-incompatible change to a protocol?
>> >
>> > - Rename the globals or the protocol file? This would make people think
>> >   twice about the change as it would incur lots of churn in the code.
>> >   The churn shouldn't be a problem for review, since we wouldn't review
>> >   experimental implementations carefully. But it's still churn.
>>
>> The way I've done it and also the way we did with pointer gestures is to
>> prefix all of the interfaces with an underscore, with a note in the
>> global interface's description describing the limitations and lack of
>> backward compatibility as well as future intentions. See here[0] for
>> exact wording.
>
> Doesn't leading underscore touch the "reserved names" of the C spec or
> something?
>
>> I think we should consider doing this to on all our experimental
>> protocols in weston (either that, or move them to wayland). And we
>> should probably move out the private protocols to a separate directory
>> as well.
>
> Perhaps.
>
>> > - Use the mechanism tried with xdg-shell? A request to make-or-fail a
>> >   specific experimental version. This seemed like a good idea, we just
>> >   have to document it clearly and make sure it's followed properly.
>> >   E.g. the make-or-fail request *must* always be opcode 0. We could
>> >   have helpers for that.
>>
>> No. That mechanism doesn't work, because it is not possible to discover
>> what version is implemented. A client that supports both xdg_shell and
>> wl_shell have no way of detecting (except try - be killed - try again)
>> whether it can use xdg_shell or wl_shell. The above described approach
>> does not have this limitation.
>
> When I read your earlier comment, I thought make-or-fail was exactly
> what you were describing. I stand corrected.
>
> Instead you want to change the interface version semantics of the
> advertisement. I'm not sure I like that or not. But it's just details...
>
> The danger with it is that people might confuse the experimental
> versioning scheme with the normal versioning scheme, and assume that
> also stable protocols must always be bound at the exact advertised
> version. This would hopefully be solved with documentation.

I agree, I think it is fair to say "compositors will only implement
the latest version of unstable protocols as known to them" somewhere
in the docs.

>
>> > Btw. I have been thinking that when one stabilizes a protocol extension
>> > and moves the .xml file into the Wayland repository, we should also
>> > always rename the global interfaces and the .xml file. This would be
>> > the last protocol break, that allows us to remove any make-or-fail
>> > requests while also ensuring no-one will accidentally use it with old
>> > code, and it won't clash with any copies of the experimental xml file.
>>
>> I agree. This is something I've taken into account when writing and
>> reviewing new protocols (as described above). But, what would be the
>> point of renaming the XML file?
>
> So that build scripts could not accidentally use an XML file from
> a wrong directory and it would make it obvious which XML file is
> experimental. That's all, I think.
>
>> The other part I brought up was adding to stable protocols. Currently
>> being discussed are additions to wl_data_device_manager (DND actions),
>> wl_seat (release request), wl_pointer (axis sources), wl_touch
>> (touch "focus" changes). Who are in charge of allowing such changes?
>> Some are quite huge (DND actions) and without a consensus, while the
>> other are relatively small, and thus with a consensus easier to reach,
>> but who's responsibility is it to make the call?
>
> Whoever is a known expert on that area.
>
> Who decides who is an expert? Uhh...

It wouldn't be good if every maintainer thinks like that :), even
worse if that results in no action taken... I understand the
difficulties of knowing/picking such go-to person, or becoming an
expert yourself on whatever topic there is at hand, but I would like
to point out the vicious circle where slow review lead to "becoming
trusted" taking ages, so the amount of people that have a say remain
static and overburdened.

Anyhow, being probably desktop-biased here, I think the rightful first
questions to ask around additions to stable protocols are:

- Can this be done through an external protocol?
- Does the lack of this feature hinder adoption or feature parity? to
which extent?

The reply to these should give an idea of how apt is a change to the
stable protocol, and the celerity required.

>
> Technically, people with push rights make the final decision to push or
> not. I'd love it if it was easy to decide what to push and what not,
> but it isn't and I don't think it can be as everyone evaluates everyone
> else differently and based on the topic at hand.

It sounds like a lack of definition of "consensus" :)

>
> Would you like to push more to Wayland and Weston?
> What do you need to be able to do that?
>
>
> My view is Weston-centric as I don't work on any other compositor. My
> biased preference for not-desktop-specific extensions that aim for
> Wayland core is to be developed in Weston, be that in topic-branches or
> otherwise. Topic-branches would get the stuff off the mailing list
> early, and merging a topic-branch to master would correspond to what we
> currently have with landing patches to master.
>
> I have a feeling, that we're trying to engineer a solution to a
> political and time management problem. I'm not sure that having

And that's I think the root of the problem. Having different agendas
is understandable, however it's IMO a maintainer duty/chore to
coordinate the intents of other contributors, or at least grasp their
priorities. If things fall out of the span of time/attention/expertise
available, we get the current situation where quite a few threads in
the ML lack direction or plainly get poor S/N ratio.

Cheers,
  Carlos