Developing a core shell protocol

Tue Feb 12 07:13:16 PST 2013

On Tue, Feb 12, 2013 at 2:52 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> On Tue, 12 Feb 2013 08:51:26 +0100
> Kai-Uwe Behrmann <ku.b at gmx.de> wrote:
>
>> On 11.02.2013 11:06, Pekka Paalanen wrote:
>> > I want to make sure again, that when we talk about shells in Wayland,
>> > we usually talk about public shell interfaces in the protocol. Therefore
>> > all of Linux desktop, regardless of DE or its components (KDE, Gnome,
>> > gnome-shell, kwin, ...) are under the one and same shell for desktops.
>> > This is about the generic shell interfaces towards all applications, of
>> > course, that can somehow make sense on any DE.
>>
>> Not exactly. Android, Ubuntu and KDE publicly aim to work on all form
>> factors. How will a desktop only wl_shell, as you outlined, serve those
>> projects concepts?
>
> I can imagine two choices there:
>
> A. Have separate, form-factor based shell interfaces, that will make
> applications adapt to the fullest for the form factor in use. Pro:
> top usability and integration. Con: you actually have to implement each
> shell interface in every compositor and toolkit (toolkit, not
> application, as toolkits should be able to abstract the form factor
> towards app code).
>
> B. Just use wl_shell, the desktop shell interface, but try to adapt its
> compositor side implementation to each form factor. Toolkits and
> applications would not be directly aware of the form factor at all.
> Pro: just one version of toolkit code needed, so all applications
> will just work somehow. Con: usability might not be that
> good, integration will suffer.
>
> There's also the third option: A+B. If it makes some sense to run
> desktop applications on the system of your choice, your compositor can
> implement wl_shell in addition to the specific shell interface.
> Applications written to support the specific form factor will use the
> specific shell interface, and other will use wl_shell. All apps will
> work, some better than others.

I think this is what I'm ultimately going to end up doing.  I'm
probably going to implement wl_shell for fallback for badly written
apps along with some sort of touch shell and probably.  One big
question here is how, when we export both wl_shell and wl_phone_shell,
do we tell applications which one to prefer and ensure that they don't
try and bind both?

> I agree with David Herrmann. Can you really write a good application
> without caring about the form factor at all? Or is it possible to
> abstract the form factor issues in the toolkit well enough, so that
> application code would be completely form-factor independent?

No, I don't think you can.  Probably the most brilliant thing Apple
did when they created iOS and the iPhone was to realize and embrace
the fact that a phone and a computer are different devices.  I don't
think we should follow suite with the rest of the world in making the
bad assumption that you can.  Microsoft's new OS makes desktop users
who want to use it with a mouse second-class citizens.  (And no,
Microsoft, making the buttons bigger does not make an app
touch-ready.)  I have a friend who has an Asus Transformer tablet and
it makes a great tablet but a not-that-great laptop because everything
is too touch-centric.

I think it's better to recognize that phones and desktops are
different and encourage developers to treat them differently in their
applications.  That being said, it would be nice to see toolkits also
recognize this and start building in things to make cros-device
development more practical.

> The above is all my speculation. I have no idea how those projects aim
> to achieve the goal of working on all form factors. Knowing how they
> intend to implement it might help with our shell discussion. Do you
> have any insight on what they are doing?

I can't tell you how iOS and Ubuntu are doing it, but I do know
something about android.  On android, UI elements are treated as
completely disposable.  Their XML system allows you to build very
modular UI's and then, based on the configuration of the device,
quickly put together the proper UI for that configuration.  If the
configuration ever changes, Android completely throws away the UI and
tells your app to create a new one for the new configuration.  For
example, if you rotate the phone, the old portrait interface is thrown
away and a new landscape version is created.  There are a lot of
different criterion that you can use to pick the UI elements such as
orientation, screen resolution, screen density, etc.

One thing we could do (with the help of toolkit people) is to allow
apps to detect when they are started what kind of device they're
running on and make their UI based on that.  I'm not sure if, given
the current toolkit climate, switching UI's without restarting the
application is feasible, but that would be nice too.

I used to think they were crazy for doing things that way, but it's
grown on me.  One thing it does is to force developers to understand
that not all devices are created equal.  People have criticized
android for trying to run on too many different devices, but it
manages to do so surprisingly well.  However, that doesn't mean I want
to run Android on my desktop.  This is because, while the apps adapt
to different screen configurations, the core OS concepts don't ever
change.  For instance, multitasking is terrible regardless of what
device you're on.

>> To illustrate a bit: attaching a monitor or TV, keyboard and mouse to a
>> phone is all what is needed for a desktop. That is long on the software
>> agenda. The difference for phone, TV, desktop computers and so on is
>> fading away quickly. These devices will more and more run on similar
>> computer hardware and their CPU/GPU fits already in a USB stick.
>
> Right, so this is about changing the form factor in flight, assuming
> one does not want to restart applications or launch a completely new
> instance of the DE. In that case I can see only option B as feasible.

Maybe, see above.

I would also note that, while the difference in terms of primary
hardware is getting smaller, the UI differences aren't.  That's why I
think we need to be flexible.

> I still think that we have not defined the scope of this discussion
> well enough. The difference between a traditional desktop with a
> keyboard and a mouse, and a touchscreen device with a virtual keyboard
> as needed etc. is not that big, when compared to a TV/set-top box. And
> like you said, the difference is only becoming smaller. What
> differences do we have to care about in the end? What
> differences actually matter to applications? Should this discussion be
> limited to desktoppy devices and exclude others? Phone, TV and laptop
> could all be desktoppy, if it makes sense.

This started out with asking if there is a good lowest common
denominator and I think that's been concluded with a resounding "no".
The more I think about it, the more I think that will just encourage
developers to make lowest common denominator apps that will be
terrible for everyone.  At this point, I think we're looking at, given
the fact that not everything is a desktop, what can we do to try and
handle multiple devices.  I would love to see some sort of
hot-plugging, but I'm not sure that's feasible.

Concerning the defining factors of phones/tablets vs. desktops:  The
more I think about this, I think there are really two major defining
factors:  the fact that everything runs maximized, and the fact that
there may be no traditional mouse.  Ever since the Xerox Star and the
Apple Lisa, GUI's have assumed that you have windows and you click on
them with a mouse.  Getting rid of these two things makes a
surprisingly large difference.  Not having the fine precision of a
mouse requires you to make interfaces that are much simpler.  It also
requires you to re-think workflow because GIMP's windows everywhere
approach no longer works.

> Also, wl_shell interfaces are still in their infancy and need more
> features even for the traditional desktop. It just might sensibly grow
> to support all these different desktoppy scenarios. Afterall, the
> obvious and naive approach is to just implement wl_shell on everything,
> where you can install third party software.
>
> Should we turn all devices into desktops hardwarewise, or work on the
> devices' terms?

No, not everything is a desktop.

> I'm afraid I lost the primary question. :-)
>
>
> Thanks,
> pq