Developing a core shell protocol

[Jason Ekstrand]

On Mon, Feb 11, 2013 at 4:06 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> On Sat, 9 Feb 2013 12:20:57 -0600
> Jason Ekstrand <jason at jlekstrand.net> wrote:
>
>> Wayland Devs,
>> This is a brainstorming e-mail concerning trying to develop some sort
>> of core shell protocol.  I will shortly begin work on developing a
>> touch-based compositor for phones and tablets and there have been
>> multiple discussions on IRC concerning trying to implement tiling
>> WM's.  There are probably several more use-cases that do not fall into
>> these lists.
>>
>> The problem with the current protocol is that wl_shell is, in the
>> words of krh, "inherently desktopy".  I believe the intention is that
>> different kinds of compositors will simply implement different
>> protocols for putting a surface on the screen.  However, many of these
>> different types of compositors are still shells in the sense that they
>> provide a mechanism for users to run other applications.  Right now,
>> if an application is written without knowledge of your specific shell
>> type, it won't work.  This is going to cause massive problems for
>> alternate shells because old or lazy software won't run on them unless
>> they try and implement the desktop protocol which is going to be a
>> hack.
>>
>> In order to solve this problem, I am suggesting that we effectively
>> split wl_shell into two components: a core shell interface, and a
>> desktopy shell interface that extends the core shell interface.  Then
>> we could write a set of other shell interfaces that each extend the
>> core shell interface and provide additional functionality not provided
>> by the core.  This way, if an application is written without knowledge
>> of a particular shell, it can use the core as a fallback to at least
>> display and work to a certain extent.  Please note that I am referring
>> to compositors that act as shells allowing the user to run external
>> programs.  Special purpose compositors such as login manager are
>> probably still going to have to be a special case.
>>
>> If this is to happen, there are a number of questions that need to get
>> answered (I'm sure this list is incomplete):
>>
>> 1) What things are common to all shells?  In order to answer this, I
>> think we need to look at a lot of use-cases such as standard desktop
>> WM's; touch-based WM's such as Android, iOS, or ModernUI; and tiling
>> window managers.  If anyone knows of other eclectic shell schemes,
>> those should go on the table too.
>
> I think that list is still quite desktop'y. We also need to consider
> phones, IVI-systems, aeroplane entertainment systems, set-top boxes and
> TVs, game consoles... anything that has a graphical display, really.
>
> A TV or a set-top box is a nice example, if you think you only have a
> remote control with some buttons, and no mouse, real keyboard, or touch
> input.
>
> At least I do not have enough experience in any one of these to be able
> to really suggest anything.
>
>> 2) Is it even feasible to try and make a lowest-common-denominator
>> shell interface?
>
> I have no idea, but it will be very difficult. I would tend to think
> that the Wayland core protocol already is such.
>
> I do not think we would want to end up with an overly generic protocol
> extension, where each environment will use only selected bits and not
> others, replacing some bits with shell-interface specific bits.
>
>> 3) What are the UI implications of an application only handling the
>> core shell? How do we make that application work in a sensible way
>> without the application providing the shell all the information it
>> would like?
>
> You'd need to define a "generic window" in all shells, and how it
> behaves, I guess... so it'll be shell specific, and can that really
> work? I'm very sceptical.
>
>> 4) What about decorations? Right now, wayland always assumes
>> client-side decorations and the client simply decorates top-level
>> windows. However, this leads to a number of problems.  Tiling WM's for
>> instance don't decorate the same way others do, and a touch-based WM
>> might not decorate at all.
>
> Decorations are quite desktop-specific, don't you think?

Exactly, they are desktop-specific.  Right now, clients assume they
have to draw decorations and most always will.  The bigger issue (this
came up on IRC the other day) is how do you tell them not to?  This
isn't an issue of you're using a completely separate interface (call
it wl_phone_shell for example) because the client could compensate.
However, if you're simply going to break the desktop protocol a little
(ex. tiling), this becomes more of an issue.

> On some systems, like TVs, they might not make any sense. I mean, even
> having a toggle for decorations.
>
>> 5) How do we handle the migration path? Do we simply add another
>> wl_core_shell interface, or do we deprecate the desktoppy parts of
>> wl_shell and add wl_desktop_shell to implement those?  Really, this is
>> probably the last question to answer.
>
> Yeah, we can punt this until we have a plan for what might work even
> without backwards compat.
>
>> Giving it a brief look, I think the following can probably stay in the "core":
>>  * ping and pong
>>  * set_toplevel
>
> How can you have multiple top-level surfaces on a game console or TV?
My understanding is that set_toplevel simply tells the compositor that
it's a main window.  In the case of a console or TV, it would get
maximized.  On the other hand, you may want to pop a window up on top
to inform the user of something which might be a different kind of
window.  Maybe I'm thinking too X/server-side, but a popup/transient
window still seems useful in this case.

>
>>  * set_transient (although positioning might not work exactly the
>> same. we need to think about that)
>
> This becomes sub-surface, if you make it shell-agnostic.
>
>>  * set_fullscreen
>
> Not needed, if the definition of top-level is fullscreen, like maybe in
> phones.

The distinction I see here is one between maximized and fullscreen.
In Android, everything is maximized, but not everything is fullscreen.
 A fullscreen app is usually a game or a video and, in that case, not
even the system decorations (clock, battery life etc.) are showing.
However, perhaps having shell-level navigation components at all is a
bit too much of an assumption.

>>  * set_name
>>  * configure
>>
>> and the following probably are specific to the desktop shell:
>>  * move
>>  * resize
>>  * set_maximized
>>
>> Note: When I say wl_shell above, I also include wl_shell_surface, as
>> the two interfaces are inherently tied together.  Each shell interface
>> is going to be accompanied by a corresponding surface interface.
>
> Right.
>
> I want to make sure again, that when we talk about shells in Wayland,
> we usually talk about public shell interfaces in the protocol. Therefore
> all of Linux desktop, regardless of DE or its components (KDE, Gnome,
> gnome-shell, kwin, ...) are under the one and same shell for desktops.
> This is about the generic shell interfaces towards all applications, of
> course, that can somehow make sense on any DE.
>
> When we move to a different shell, desktop features largely stop making
> sense.
>
> http://cgit.freedesktop.org/wayland/weston/tree/notes.txt tries to
> explain the distinction between a shell and a shell from the
> perspective of a window manager developer.
>
> Therefore, I would make a counter-question: are you seeking to find the
> common denominator over all graphical environments you can still call
> "desktop", even if they have very different GUI paradigms like a tiling
> WM (right?), or are you truly seeking for a common denominator over all
> shells, which includes TVs, phones, and wristwatches?

> Or in other words, are you looking for the common things in all
> applications that can run on a desktop, or are you looking for an
> interface which allows you to run an application written for a desktop
> in a TV, with a regular TV remote as input?

Perhaps I should be asking a better question.  That question would be,
How should alternate compositors be handled?  Perhaps the best answer
isn't to change wl_shell, but to simply break it in a reasonable way
such as ignoring requests by the client to move the window.  I guess
my concern is that, since having to constantly break the X spec is
something we're trying to get away from, is breaking the Wayland spec
really the right way to do this?  Is completely ignoring move/resize
requests actually breaking things?

My primary use-case is android which covers phones, tablets, and TV's.
 Probably the most defining characteristics of those environments is
lack of a mouse and the fact that every window is maximized (except on
the Galaxy Note II).  I would like clients written in GTK or Qt to run
without huge modifications if the user so wishes.  (Obviously, they
would need to be touch-aware or you would need a bluetooth mouse.)  Is
that going to encourage too many broken clients?

>
> Thanks,
> pq