[PATCH weston 0/6] ivi-shell proposal

Pekka Paalanen ppaalanen at gmail.com
Sun Sep 8 02:31:56 PDT 2013

On Sun, 08 Sep 2013 00:13:55 +0900
nobuhiko_tanibata <nobuhiko_tanibata at localhost.xddp.denso.co.jp> wrote:

> 2013-09-06 19:16 に Pekka Paalanen さんは書きました:
> > On Fri, 6 Sep 2013 08:39:24 +0900
> > "Nobuhiko Tanibata" <NOBUHIKO_TANIBATA at xddp.denso.co.jp> wrote:
> > 
> >> ----- Original Message -----
> >> From: "Pekka Paalanen" <ppaalanen at gmail.com>
> >> To: "Nobuhiko Tanibata" <NOBUHIKO_TANIBATA at xddp.denso.co.jp>
> >> Cc: <wayland-devel at lists.freedesktop.org>; <securitycheck at denso.co.jp>
> >> Sent: Thursday, September 05, 2013 10:02 PM
> >> Subject: Re: [PATCH weston 0/6] ivi-shell proposal
> >> 
> >> 
> >> > On Wed, 4 Sep 2013 09:08:26 +0900
> >> > "Nobuhiko Tanibata" <NOBUHIKO_TANIBATA at xddp.denso.co.jp> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> This series implements ivi-shell to fulfill use cases of In-Vehicle
> >> >> Infotainment, IVI. Such use cases are well overviewed in a project;
> >> >> Genivi IVI layer management.
> >> >> http://projects.genivi.org/ivi-layer-management/node/13
> >> >>
> >> >> A motivation of this series and basis idea are introduced by Ossama
> >> >> at Automotive Linux Summit 2012 spring. The series implements
> >> >> ivi-shell part. Additionally, GENIVI LM Client Library at slide 20 is
> >> >> contributed to ivi-layer-management project to support compatible
> >> >> interfaces for Genivi Layer management users.
> >> >> http://events.linuxfoundation.org/images/stories/pdf/als2012_othman.pdf
> >> >>
> >> >> Before I start implementation of ivi-shell, Core members of Genivi
> >> >> IVI layer management defined draft of ivi-shell.xml to fulfill
> >> >> requirements of IVI layer management, inviting Kristian. The series
> >> >> also includes the ivi-shell.xml with updates I faced in actual
> >> >> implementation.
> >> >>
> >> >> Please give me any suggestions.
> >> >
> >> > Hi,
> >> >
> >> > I have understood that the whole design has been cut deep into stone a
> >> > long time ago. What are you able to change now? I do not think it is
> >> > worth commenting on things you can no longer change, so what aspects are
> >> > you looking suggestions for?
> >> >
> >> 
> >> Hi,
> >> 
> >> I would like to get feedback about that if somebody has a similar 
> >> motivation
> >> to support ivi as well as desktop and tablet.
> >> So it is not a stone, just a proposal. If somebody has good idea, I 
> >> would
> >> like to use it or collaborate it.
> > 
> > Ok, I just had the understanding that the layer manager simply has to
> > be a separate process and not built into the compositor. If that is
> > not the case, then that is very good news indeed. Everything that
> > manages surfaces, layers, windows, or whatever belongs into the
> > compositor process, where they are much easier to implement and you
> > don't need to introduce interfaces and IPC which are later hard to
> > develop further and cause latencies. Roundtrips and synchronous
> > calls between processes can become a difficult bottle-neck, as X11
> > has taught us. Also having too many processes becomes a real mess
> > when trying to avoid deadlocks but still keep things coherent and
> > glitch-free (see X11 server vs. window manager vs. ...).
> > So I'm roughly on the same track as Andreas Pokorny.
> > 
> > Weston should become the window manager and layer manager, while
> > weston backends deal with the hardware details of compositing. For
> > example, the Raspberry Pi backend of Weston forwards all
> > compositing to the VideoCore firmware, unlike any other backend
> > who actually render a composite themselves. However, the rpi
> > backend does not forward all surfaces to VideoCore all the time,
> > but only the visible ones as needed. And Weston is the only
> > component that *can* know what is visible at any time, since Weston
> > contains the complete scene graph. Weston's internal architecture
> > is also well-suited for *automatically* deciding how to use the
> > limited hardware resources like overlays efficiently, and fall back
> > as necessary, per each output frame. You cannot do that, if you
> > tell applications about specific hardware resources.
> Yes, my understind is the same as yours.

Great. :-)

> And I expects Weston backend 
> you mentioned next paragraph is ideally released as reference from SoC 
> vender.
> For exmple, some SoC vendor supports 2D blit engine which can effciently 
> composite surfaces and relese weston backend as reference.

Tbh, I doubt vendor's abilities in producing a proper Weston
backend, and maintaining it as Weston changes. But yeah, something
like that.

> > I suspect there might be some terminology differences here.
> > Something like what IVI calls a "compositor" is a "weston backend"
> > when talking about Weston and Wayland, and IVI layer manager is
> > actually a window manager which is just a shell plugin to Weston.
> > Or am I completely off?
> > 
> Yes, you are correct. And I focus on shell part to realize ivi 
> requirements.

Shell is usually the part which is primarily used by applications, not
supporting components of a graphical environment (see wl_shell vs.
desktop_shell protocol interfaces). I mean the public part of
shell, like the wl_shell interface for desktop applications.

Which brings me to another question: how likely is it that you
actually want to support PC desktop applications unmodified
directly on an IVI system? And I mean on the main compositor of an
IVI system, which I would imagine to be quite a critical component.

I think native IVI applications could be different enough to PC desktop
applications, that creating a new shell interface to *replace*
wl_shell (or whatever shell interface we will be using in the
future for PC desktop) would be a right choice.

If you actually do want to support PC desktop applications, I could
see you having another, nested compositor just for supporting PC
applications, which could run on a more restricted environment and
maybe access to a more complex GPU which would be too risky for
the main compositor in IVI. A sort of "untrusted" domain.

Well, Genivi probably has already designed all that, so I'm just
reinventing the wheel badly here.

> > Have you tried to map your IVI concepts of surface/layer/display to
> > Wayland wl_surface, wl_subsurface, and wl_output? I don't really
> > see what kind of interfaces your applications (Wayland clients?)
> > expect to use.
> Yes. As you comment, some use case; visibility and crop/scaling is not 
> supported now.
> So I thought starting new set of protocal to cover ivi requiremnts would 
> be better.
> But I will re-consider them and mail it back.

Right. I replied from purely FOSS point of view. You probably have
time and money deadlines which you must meet while creating a
self-standing product, and in that case, I do understand going with
a big re-invention. I understand it, but I'm not happy about it,
although if you can publish your ad hoc approach like you do here,
you are contributing valuable experience to the community.
Especially, if you say something about the shortcomings of the
design and use experiences.

I'm very happy to see this proposal on the mailing list, even when
I do not agree with it.

> > When I look at the protocol in ivi-shell.xml, I get the feeling that:
> > - Interfaces ivi_layer, ivi_controller_surface,
> >   ivi_controller_layer, ivi_controller_screen, and ivi_controller
> >   should be internal implementation details inside the weston
> >   process, not protocol. Having these as interfaces looks like the
> >   X architecture, where the X server process and window manager
> >   process continuously struggle to keep each other up-to-date, and
> >   carefully try to keep state in sync (and fail), which also makes
> >   races and glitches practically unavoidable.
> I have basic question to the above; strugling. wl_subsurface supports 
> set_poistion now.

set_position for sub-surfaces is *always* relative to the parent
surface. The sub-surface position is given as a point on the parent
surface. You still have no control where on the output any surface
will appear, because you cannot control where the root surface of a
tree of sub-surfaces will appear.

If a parent surface is moved on screen, all its sub-surfaces stay
glued to it automatically without any client interaction.

> I thought it implies positioning from a windowmanger is allowed on 
> weston basic concept.

From a window manager, yes, but this assumes that the window manager
is in-process with weston; the window manager must be a compositor
plugin. Making it an external process will be a huge amount of
trouble and performance loss.

And the protocol you propose seems to be for an external window
manager process, from my understanding.

> Each other up-to-date may occurs from window manager and weston internl 
> decision.
> Or positioning of sub surfaces is out of scope of weston and it just 
> composite them according to attributes.
> I may be wrong. Please let me know history.

The first thing is that in Wayland core protocol, there is no
global positioning system. There are no global coordinates in the
protocol. All coordinates that clients deal with, are relative to
some wl_surface. (This also allows the compositor to do arbitrary
transformations on surfaces, because there is no need for clients
to know about them, and so no need to express transformations in
the protocol.)

Sub-surfaces do not change that design.

Another thing is that with sub-surface positioning, the information
flows strictly into one direction, and we use wl_surface.commit
request on the parent surface to synchronize everything. That means
that a client can manipulate a whole tree of sub-surfaces, and
*guarantee* an atomic, flicker-free, glitch-free update on screen.

If one needed IPC between a compositor and a window manager, you
would either risk visual glitches as compositor first draws one
thing before the window manager says otherwise, or jerky compositor
performance as it needs to wait for the window manager to respond
before it can draw anything. I don't see any way around that.

> > - Interface ivi_client is just a reinvention of wl_compositor and
> >   wl_subcompositor.
> > - Interface ivi_surface is a reinvention of wl_surface.
> > 
> > Yes, I see there are some details to may want to control like
> > surface opacity, that the current Wayland protocols do not support,
> > but I don't think that replacing everything is a good way to start.
> > 
> > It is also very hard to see how objects from all these interfaces
> > are created, and how (if?) they associate to any other protocol
> > objects.
> > 
> > Btw. if you need support for surface scaling and cropping, there
> > have been discussion on the Wayland mailing list to bring a crop &
> > scale protocol extension to Wayland. It is actually necessary for
> > efficient video playback etc., so pushing that forward would be
> > nice.
> > 
> Thank you. I will check.

Search for "wl_scaler", that was the working name of a proposal.

> > After looking through the two links you gave, the ivi-shell.xml,
> > and what you have wrote in the emails, I still have no clue what is
> > the big picture here.
> > 
> I will draw a pciture to explain them. I will mail it back later.

Cool, thanks.

I saw the terms surface, layer, etc. in the IVI docs but I didn't
really get what they are used for.

> > - What processes are going to use which interfaces? It looks to me
> >   like some interfaces are not meant for all Wayland clients, but
> >   how is it supposed to work?
> > 
> > - What components are in a whole IVI system, from the point of view
> >   of Wayland protocol? What are the responsibilities of each
> >   component and how are these distributed into processes?
> > 
> > - What does a typical IVI application do in terms of Wayland
> >   protocol? Are you using wl_compositor at all? Or any other
> >   Wayland core interfaces?
> > 
> > These are just few questions to get you oriented on what kind of
> > things puzzle me here. Obviously, I have never been in touch with
> > Genivi stuff before, and I would assume most here have not either.
> > 
> > The protocol you propose seems to have many references to
> > "id_native" and "native content", what is all this "native" stuff
> > about? Or all the integer id's you seem to be sending back and
> > forth, why can't you use real protocol objects to refer to those?
> > 
> > 
> > The above is the first impression from someone, who does not know
> > anything about the IVI architecture, but is fairly familiar with
> > Wayland. Sorry if it came out harsh, but I feel like I totally
> > missed the whole background of this proposal: why design it like
> > that?
> > 
> Thank you again for good feedback. They are very helpfull for me.

Thank you,

More information about the wayland-devel mailing list