[PATCH weston 0/6] ivi-shell proposal

[Jason Ekstrand]

On Sep 13, 2013 8:35 AM, "nobuhiko_tanibata" <
nobuhiko_tanibata at xddp.denso.co.jp> wrote:
>
> Hi Michael-san,
>
> As Pekka and Jason suggests in feedback, shall we clearly explain two
topics before stepping in details.
> - what ivi-shell tries to standardizes
> - why ivi-controller tries to conrol surfaces and layers in separated
process from a compositor.
>
> @Pekka and Jason,
> Thank you for many feedbacks. does it make sense about two topics to be
clarified at first?

Yes, I think that sounds like a good idea.
--Jason Ekstrand

>
> Best regards,
> Tanibata
>
>
> 2013-09-12 20:13 に Pekka Paalanen さんは書きました:
>
>> On Mon, 9 Sep 2013 08:01:51 +0000
>> <Michael.Schuldt at bmw.de> wrote:
>>
>>> Hi all,
>>>
>>> let me give some technical background on the extension, and why we
>>> have decided to design the first approach of the extensions like it
>>> is published from Tanibata-san. In general, we would like to get the
>>> feedback on the extensions, to get it fitting in the FOSS approach,
>>> fulfilling the ivi requirements. See my comments in line.
>>
>>
>> Jason Ekstrand had an excellent point asking which parts you are
>> actually looking for to standardize.
>>
>>> >> Hi,
>>> >>
>>> >> I would like to get feedback about that if somebody has a similar
>>> >> motivation to support ivi as well as desktop and tablet. So it is
>>> >> not a stone, just a proposal. If somebody has good idea, I would
>>> >> like to use it or collaborate it.
>>> >
>>> > Ok, I just had the understanding that the layer manager simply has
>>> > to be a separate process and not built into the compositor.
>>>
>>> Not really. Currently in the compliance from genivi, we have the
>>> separation of Graphic Backend Server and LayerManagement. But this
>>> should now change we want to skip that approach and want define a
>>> protocol + ivi extensions, including a reference compositor. We have
>>> decided to use wayland + ivi extension as compositor protocol and
>>> weston as the reference implementation for that.
>>
>>
>> Very cool!
>>
>>> > If that is
>>> > not the case, then that is very good news indeed. Everything that
>>> > manages surfaces, layers, windows, or whatever belongs into the
>>> > compositor process, where they are much easier to implement and you
>>> > don't need to introduce interfaces and IPC which are later hard to
>>> > develop further and cause latencies.
>>>
>>> Sure that is the technical best fitting solution for that, we
>>> do not want to use separate IPC for the composition approach.
>>> > So I'm roughly on the same track as Andreas Pokorny.
>>>
>>> We too, but unfortunately it is not possible all the time to get
>>> the complete user interface included in the compositor itself. We
>>> have to guarantee fast startup time to get early video running in 2
>>> seconds. On different systems we can not achieve that with complete
>>> HMI-UI is up.
>>
>>
>> Oh, no, I am not suggesting to put user interfaces in the compositor,
>> not at all. Only the window management, which means window positioning,
>> z-order, etc. For instance, in Weston virtual desktops are implemented
>> in the compositor, and not by an external process juggling window lists
>> via IPC and setting visibilities and positions.
>>
>> Proposing just like with weston and desktop-shell, there is the weston
>> plugin called desktop-shell.so which does window management, and then
>> there is the helper program weston-desktop-shell, that does all the UI
>> drawing in a separate process.
>>
>> Like Jason Ekstrand explained, desktop-shell.so and
>> weston-desktop-shell communicate with each other not only using the
>> core Wayland protocol, but also using a private (or proprietary)
>> protocol extension.
>>
>> While the basic IPC mechanism is all the same (Wayland), there are
>> different parts: extensions. The desktop-shell extension is actually a
>> protected one, only weston-desktop-shell as started by desktop-shell.so
>> can access it.
>>
>>> > Have you tried to map your IVI concepts of surface/layer/display to
>>> > Wayland wl_surface, wl_subsurface, and wl_output? I don't really
>>> > see what kind of interfaces your applications (Wayland clients?)
>>> > expect to use.
>>> >
>>> > When I look at the protocol in ivi-shell.xml, I get the feeling
>>> > that:
>>> > - Interfaces ivi_layer, ivi_controller_surface,
>>> >   ivi_controller_layer, ivi_controller_screen, and ivi_controller
>>> > should be internal implementation details inside the weston
>>> > process, not protocol. Having these as interfaces looks like the X
>>> > architecture, where the X server process and window manager process
>>> > continuously struggle to keep each other up-to-date, and carefully
>>> > try to keep state in sync (and fail), which also makes races and
>>> > glitches practically unavoidable. - Interface ivi_client is just a
>>> > reinvention of wl_compositor and wl_subcompositor.
>>> > - Interface ivi_surface is a reinvention of wl_surface.
>>>
>>> Let me explain, why this separation occurs.
>>> From the ivi perspective we have separated the different applications
>>> in ivi-client, ivi-control.
>>>
>>> ivi-client: Is just a client, like browser, navigation which is
>>> embedded in the browser. That belongs to the ivi-surface. These
>>> process have really limited access the can just create the surface
>>> and that's all. The only important point which the want to know is if
>>> the visibility has changed and the size to adapt the rendered
>>> content.  A navigation should not render the content if it is not
>>> visible or assigned to a layer.
>>
>>
>> That is actually very well fitting to what we already have in Wayland.
>> Just replace ivi-client with a wl_client (that's not a protocol
>> object, it represents a display connection from a client process), and
>> ivi-surface with wl_surface+wl_shell_surface.
>>
>> The limited access is already there: in Weston, clients are free to
>> create any number of wl_surfaces. However, no surface is shown
>> anywhere on screen, unless it is assigned a suitable role. Examples of
>> roles are: "pointer cursor", "top-level window", "drag icon", and
>> "popup (menu)". Without a role, the compositor cannot know what to do
>> with a surface.
>>
>> "top-level window" and "popup menu" are shell concepts. If you replace
>> wl_shell with your own ivi_shell, you could have roles like: "TV
>> output", "rear camera", "speedometer". I'm totally pulling these out of
>> my hat, I don't know what actually makes sense for you.
>>
>> Then, your shell module in weston could handle such surfaces as you
>> want. For example, if you get a surface with the role "TV output", the
>> shell plugin could show that surface cloned on all entertainment
>> monitors. Or whatever.
>>
>> Or if something signals that now would be a /really/ good time to show
>> the rear view camera, then your shell plugin can make it so, if there
>> is a surface with such role assigned.
>>
>> Now, something like a "rear view camera" sounds like a pretty important
>> role for a surface, so you may want to make it privileged. You don't
>> want any random application to claim to be a rear view camera. In that
>> case, you can simply add whatever authentication you need to do in the
>> shell plugin. For example, only this one process started directly by
>> the shell plugin is allowed to claim the rear view camera role. Anyone
>> else trying that will get killed. On the protocol design, such
>> protection is pretty irrelevant. All you need is for the shell plugin
>> to know who is authorized for what, and it can then enforce the
>> restriction, whether the privileged role is requested by a method in a
>> otherwise public protocol extension or in its own extension.
>>
>>> ivi-controller : This is the main application which is controlling
>>> the compositor like the UI. That could be the hmi, but it can also
>>> some debug applications and application which is triggering a screen
>>> shot of surfaces, layer, displays. If you driving a car during the
>>> development you want to get some screenshots which are normally
>>> triggered by an external application like the log and trace framework.
>>
>>
>> Ok. I am proposing to have window management in the compositor plugin.
>> But, you can offer additional debugging interfaces in the protocol.
>>
>> For example, Weston does not allow clients to control *any* input
>> devices. Clients simply cannot feed fake input events to other clients.
>> But for testing and debugging, we need a way to fake input from a
>> client. The solution: a test plugin.
>>
>> The test pluging, which is loaded by the compositor only for testing
>> (in your case debugging) purposes registers a new protocol extension:
>> the test interface. That test interface then allows to do all kinds of
>> ugly things from clients.
>>
>> I'm not sure what you mean by "UI", but we separate controlling windows
>> from drawing content, like I explained of the desktop shell.
>>
>> As a recap, the shell plugin is what enforces and executes policies.
>> Clients only indicate intent or purpose, like "This surface is a menu,
>> and I want it to pop up right there on my window." A menu is an
>> interesting example, since it is more than just a surface role: it
>> comes with an input device grab. While the menu is up, input is
>> directed to it. But, it is the shell plugin that can decide, that ok,
>> you're done now, grab is broken, the menu surface hidden, and the
>> client is signalled that something dismissed the menu. Also, clients
>> cannot put up a menu at random times, it has to be done as a response
>> to an input event. All this prevents many kinds of client misbehaviour.
>>
>>> > Yes, I see there are some details to may want to control like
>>> > surface opacity, that the current Wayland protocols do not support,
>>> > but I don't think that replacing everything is a good way to start.
>>> > It is also very hard to see how objects from all these interfaces
>>> > are created, and how (if?) they associate to any other protocol
>>> > objects.
>>>
>>> On goal is to get setup a complete scenery during the startup of the
>>> compositor. Therefore the compositor is able to just initialize the
>>> main visible attribute during startup. That implies too that it
>>> should be possible, without the final application is running (like
>>> navigation). Therefore we need some logical objects like ivi-surface,
>>> ivi-layer. Which can be already controlled, whithout a application
>>> which holds the content is already running.
>>
>>
>> But if all layer and window management is internal to the compositor
>> process, then you don't need any of this control protocol.
>>
>> Just create some internal data structures, and when a client comes up
>> with a surface and requests a particular role, you can plug that
>> surface into the slot you prepared in advance.
>>
>> So, yes, this does imply that the compositor or the shell plugin knows
>> the scenegraph, but... isn't that exactly the compositor's purpose, to
>> gather the scenegraph and get it on screen? Clients "only" provide
>> content.
>>
>> For example, the desktop-shell protocol extension has special surface
>> roles "background" and "panel". When weston-desktop-shell client
>> assigns these roles to its surfaces, the compositor (shell plugin) will
>> immediately plug them into the correct places on screen. If the shell
>> plugin wanted to reserve screen space for them in advance, it could.
>> After all, the shell plugin is responsible for arranging everything on
>> (and off) screen.
>>
>>> > Btw. if you need support for surface scaling and cropping, there
>>> > have been discussion on the Wayland mailing list to bring a crop &
>>> > scale protocol extension to Wayland. It is actually necessary for
>>> > efficient video playback etc., so pushing that forward would be
>>> > nice.
>>>
>>> Yeah I see that, and we will consider it. I think we can redesign the
>>> approach like source and destination region to the crop and scale
>>> instead.
>>>
>>> > After looking through the two links you gave, the ivi-shell.xml,
>>> > and what you have wrote in the emails, I still have no clue what is
>>> > the big picture here.
>>> >
>>> > - What processes are going to use which interfaces? It looks to me
>>> >   like some interfaces are not meant for all Wayland clients, but
>>> >   how is it supposed to work?
>>> > - What components are in a whole IVI system, from the point of view
>>> >   of Wayland protocol? What are the responsibilities of each
>>> >   component and how are these distributed into processes?
>>>
>>> ivi-control : Is the only process which is allowed to control the
>>> compositor itself. But they need some informations about other
>>> applications which are running. In the ivi-domain, each graphical
>>> application has a unique ID which is defined by the OEM or platform
>>> provider. ivi-client :  are just other applications like browser or
>>> navigation.
>>
>>
>> Ok, so you probably need a protocol interface to tell the compositor
>> the unique ID, and perhaps an authentication/authorization interface,
>> so that a shell plugin has all the needed information to do the
>> controlling.
>>
>>> > - What does a typical IVI application do in terms of Wayland
>>> >   protocol? Are you using wl_compositor at all? Or any other
>>> >   Wayland core interfaces?
>>>
>>> We were focused on just using the wayland core interface. But we are
>>> open to extend this approach.
>>
>>
>> For the record, "Wayland core" is everything in
>> http://cgit.freedesktop.org/wayland/wayland/tree/protocol/wayland.xml
>> except wl_shell and wl_shell_surface, which are specific to desktop
>> applications. But you are free to pick and choose individual global
>> interfaces.
>>
>>> > The protocol you propose seems to have many references to
>>> > "id_native" and "native content", what is all this "native" stuff
>>> > about? Or all the integer id's you seem to be sending back and
>>> > forth, why can't you use real protocol objects to refer to those?
>>>
>>> This is a little bit confusing, the initial set of these objects like
>>> it is defined here :
>>>
>>>
http://git.projects.genivi.org/?p=layer_management.git;a=blob;f=wayland-ivi-extension/protocol/ivi-shell.xml;h=6092b13ef31e55f3db338be2e471d0e956be5b9e;hb=ivi-extension
>>>
>>> Is focused on using the ivi-surface instead of the native id. The
>>> rational behind that is to define unique ids over the complete
>>> process space. Therefore the HMI is able to control the different
>>> ivi-surface properties.
>>
>>
>> Ok, you are inventing global names for these protocol objects, so you
>> could do "window management" in a separate process, right? If you made
>> the HMI controller a compositor plugin, these global names would not be
>> needed, perhaps?
>>
>> Perhaps I still do not fully grasp what the HMI controller is. And I
>> have more of your emails to read still.
>>
>> You may have a very good reason to keep the HMI controller separate,
>> but the thing I am questioning is, is the interface towards the
>> compositor the right one. Should the controller really be responsible
>> for window management, or should it just send simple messages like
>> "rear view camera on/off" or even "gear: reverse".
>>
>> Even if the HMI controller was one big block in your architecture
>> diagrams, you could split it into a compositor plugin and an external
>> process. Then the private Wayland protocol extension they use to
>> communicate with each other will be just an implementation detail of
>> the whole block, which you might not need to standardize.
>>
>> There actually exists examples of such a design: most of the
>> Wayland-enabled EGL implementations use a private protocol extension to
>> communicate between the server and the client. The trick is, both the
>> server and the clients load the same libEGL, so the protocol becomes a
>> completely hidden implementation detail.
>>
>>> In the end we are very lucky about any input to get a final best
>>> fitting solution for both the ivi aspects and the final
>>> implementation for that in the FOSS domain.
>>
>>
>> I am very glad you came forward. :-)
>>
>>
>> Thanks,
>> pq
>
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20130913/bcf99386/attachment-0001.html>