Developing a core shell protocol

[Kristian Høgsberg]

On Mon, Feb 11, 2013 at 10:52:00AM -0600, Jason Ekstrand wrote:
> On Mon, Feb 11, 2013 at 4:06 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> > On Sat, 9 Feb 2013 12:20:57 -0600
> > Jason Ekstrand <jason at jlekstrand.net> wrote:
> >
> >> Wayland Devs,
> >> This is a brainstorming e-mail concerning trying to develop some sort
> >> of core shell protocol.  I will shortly begin work on developing a
> >> touch-based compositor for phones and tablets and there have been
> >> multiple discussions on IRC concerning trying to implement tiling
> >> WM's.  There are probably several more use-cases that do not fall into
> >> these lists.
> >>
> >> The problem with the current protocol is that wl_shell is, in the
> >> words of krh, "inherently desktopy".  I believe the intention is that
> >> different kinds of compositors will simply implement different
> >> protocols for putting a surface on the screen.  However, many of these
> >> different types of compositors are still shells in the sense that they
> >> provide a mechanism for users to run other applications.  Right now,
> >> if an application is written without knowledge of your specific shell
> >> type, it won't work.  This is going to cause massive problems for
> >> alternate shells because old or lazy software won't run on them unless
> >> they try and implement the desktop protocol which is going to be a
> >> hack.
> >>
> >> In order to solve this problem, I am suggesting that we effectively
> >> split wl_shell into two components: a core shell interface, and a
> >> desktopy shell interface that extends the core shell interface.  Then
> >> we could write a set of other shell interfaces that each extend the
> >> core shell interface and provide additional functionality not provided
> >> by the core.  This way, if an application is written without knowledge
> >> of a particular shell, it can use the core as a fallback to at least
> >> display and work to a certain extent.  Please note that I am referring
> >> to compositors that act as shells allowing the user to run external
> >> programs.  Special purpose compositors such as login manager are
> >> probably still going to have to be a special case.
> >>
> >> If this is to happen, there are a number of questions that need to get
> >> answered (I'm sure this list is incomplete):
> >>
> >> 1) What things are common to all shells?  In order to answer this, I
> >> think we need to look at a lot of use-cases such as standard desktop
> >> WM's; touch-based WM's such as Android, iOS, or ModernUI; and tiling
> >> window managers.  If anyone knows of other eclectic shell schemes,
> >> those should go on the table too.
> >
> > I think that list is still quite desktop'y. We also need to consider
> > phones, IVI-systems, aeroplane entertainment systems, set-top boxes and
> > TVs, game consoles... anything that has a graphical display, really.
> >
> > A TV or a set-top box is a nice example, if you think you only have a
> > remote control with some buttons, and no mouse, real keyboard, or touch
> > input.
> >
> > At least I do not have enough experience in any one of these to be able
> > to really suggest anything.
> >
> >> 2) Is it even feasible to try and make a lowest-common-denominator
> >> shell interface?
> >
> > I have no idea, but it will be very difficult. I would tend to think
> > that the Wayland core protocol already is such.
> >
> > I do not think we would want to end up with an overly generic protocol
> > extension, where each environment will use only selected bits and not
> > others, replacing some bits with shell-interface specific bits.
> >
> >> 3) What are the UI implications of an application only handling the
> >> core shell? How do we make that application work in a sensible way
> >> without the application providing the shell all the information it
> >> would like?
> >
> > You'd need to define a "generic window" in all shells, and how it
> > behaves, I guess... so it'll be shell specific, and can that really
> > work? I'm very sceptical.
> >
> >> 4) What about decorations? Right now, wayland always assumes
> >> client-side decorations and the client simply decorates top-level
> >> windows. However, this leads to a number of problems.  Tiling WM's for
> >> instance don't decorate the same way others do, and a touch-based WM
> >> might not decorate at all.
> >
> > Decorations are quite desktop-specific, don't you think?
> 
> Exactly, they are desktop-specific.  Right now, clients assume they
> have to draw decorations and most always will.  The bigger issue (this
> came up on IRC the other day) is how do you tell them not to?  This
> isn't an issue of you're using a completely separate interface (call
> it wl_phone_shell for example) because the client could compensate.
> However, if you're simply going to break the desktop protocol a little
> (ex. tiling), this becomes more of an issue.
> 
> > On some systems, like TVs, they might not make any sense. I mean, even
> > having a toggle for decorations.
> >
> >> 5) How do we handle the migration path? Do we simply add another
> >> wl_core_shell interface, or do we deprecate the desktoppy parts of
> >> wl_shell and add wl_desktop_shell to implement those?  Really, this is
> >> probably the last question to answer.
> >
> > Yeah, we can punt this until we have a plan for what might work even
> > without backwards compat.
> >
> >> Giving it a brief look, I think the following can probably stay in the "core":
> >>  * ping and pong
> >>  * set_toplevel
> >
> > How can you have multiple top-level surfaces on a game console or TV?
> My understanding is that set_toplevel simply tells the compositor that
> it's a main window.  In the case of a console or TV, it would get
> maximized.  On the other hand, you may want to pop a window up on top
> to inform the user of something which might be a different kind of
> window.  Maybe I'm thinking too X/server-side, but a popup/transient
> window still seems useful in this case.
> 
> >
> >>  * set_transient (although positioning might not work exactly the
> >> same. we need to think about that)
> >
> > This becomes sub-surface, if you make it shell-agnostic.
> >
> >>  * set_fullscreen
> >
> > Not needed, if the definition of top-level is fullscreen, like maybe in
> > phones.
> 
> The distinction I see here is one between maximized and fullscreen.
> In Android, everything is maximized, but not everything is fullscreen.
>  A fullscreen app is usually a game or a video and, in that case, not
> even the system decorations (clock, battery life etc.) are showing.
> However, perhaps having shell-level navigation components at all is a
> bit too much of an assumption.
> 
> >>  * set_name
> >>  * configure
> >>
> >> and the following probably are specific to the desktop shell:
> >>  * move
> >>  * resize
> >>  * set_maximized
> >>
> >> Note: When I say wl_shell above, I also include wl_shell_surface, as
> >> the two interfaces are inherently tied together.  Each shell interface
> >> is going to be accompanied by a corresponding surface interface.
> >
> > Right.
> >
> > I want to make sure again, that when we talk about shells in Wayland,
> > we usually talk about public shell interfaces in the protocol. Therefore
> > all of Linux desktop, regardless of DE or its components (KDE, Gnome,
> > gnome-shell, kwin, ...) are under the one and same shell for desktops.
> > This is about the generic shell interfaces towards all applications, of
> > course, that can somehow make sense on any DE.
> >
> > When we move to a different shell, desktop features largely stop making
> > sense.
> >
> > http://cgit.freedesktop.org/wayland/weston/tree/notes.txt tries to
> > explain the distinction between a shell and a shell from the
> > perspective of a window manager developer.
> >
> > Therefore, I would make a counter-question: are you seeking to find the
> > common denominator over all graphical environments you can still call
> > "desktop", even if they have very different GUI paradigms like a tiling
> > WM (right?), or are you truly seeking for a common denominator over all
> > shells, which includes TVs, phones, and wristwatches?
> 
> > Or in other words, are you looking for the common things in all
> > applications that can run on a desktop, or are you looking for an
> > interface which allows you to run an application written for a desktop
> > in a TV, with a regular TV remote as input?
> 
> Perhaps I should be asking a better question.  That question would be,
> How should alternate compositors be handled?  Perhaps the best answer
> isn't to change wl_shell, but to simply break it in a reasonable way
> such as ignoring requests by the client to move the window.  I guess
> my concern is that, since having to constantly break the X spec is
> something we're trying to get away from, is breaking the Wayland spec
> really the right way to do this?  Is completely ignoring move/resize
> requests actually breaking things?
> 
> My primary use-case is android which covers phones, tablets, and TV's.
>  Probably the most defining characteristics of those environments is
> lack of a mouse and the fact that every window is maximized (except on
> the Galaxy Note II).  I would like clients written in GTK or Qt to run
> without huge modifications if the user so wishes.  (Obviously, they
> would need to be touch-aware or you would need a bluetooth mouse.)  Is
> that going to encourage too many broken clients?

I think this is the core of the problem: you can't expect an
application written for the desktop UI paradigm to just run on a phone
without.  And if you want to do that anyway, I'd say you have to
provide wl_shell and somehow handle integration of desktop style apps
on the phone in your phone wm/compositor.

Kristian