On Sep 13, 2013 8:35 AM, "nobuhiko_tanibata" <<a href="mailto:nobuhiko_tanibata@xddp.denso.co.jp">nobuhiko_tanibata@xddp.denso.co.jp</a>> wrote: > > Hi Michael-san, > > As Pekka and Jason suggests in feedback, shall we clearly explain two topics before stepping in details. > - what ivi-shell tries to standardizes > - why ivi-controller tries to conrol surfaces and layers in separated process from a compositor. > > @Pekka and Jason, > Thank you for many feedbacks. does it make sense about two topics to be clarified at first? Yes, I think that sounds like a good idea. --Jason Ekstrand > > Best regards, > Tanibata > > > 2013-09-12 20:13 に Pekka Paalanen さんは書きました: > >> On Mon, 9 Sep 2013 08:01:51 +0000 >> <<a href="mailto:Michael.Schuldt@bmw.de">Michael.Schuldt@bmw.de</a>> wrote: >> >>> Hi all, >>> >>> let me give some technical background on the extension, and why we >>> have decided to design the first approach of the extensions like it >>> is published from Tanibata-san. In general, we would like to get the >>> feedback on the extensions, to get it fitting in the FOSS approach, >>> fulfilling the ivi requirements. See my comments in line. >> >> >> Jason Ekstrand had an excellent point asking which parts you are >> actually looking for to standardize. >> >>> >> Hi, >>> >> >>> >> I would like to get feedback about that if somebody has a similar >>> >> motivation to support ivi as well as desktop and tablet. So it is >>> >> not a stone, just a proposal. If somebody has good idea, I would >>> >> like to use it or collaborate it. >>> > >>> > Ok, I just had the understanding that the layer manager simply has >>> > to be a separate process and not built into the compositor. >>> >>> Not really. Currently in the compliance from genivi, we have the >>> separation of Graphic Backend Server and LayerManagement. But this >>> should now change we want to skip that approach and want define a >>> protocol + ivi extensions, including a reference compositor. We have >>> decided to use wayland + ivi extension as compositor protocol and >>> weston as the reference implementation for that. >> >> >> Very cool! >> >>> > If that is >>> > not the case, then that is very good news indeed. Everything that >>> > manages surfaces, layers, windows, or whatever belongs into the >>> > compositor process, where they are much easier to implement and you >>> > don't need to introduce interfaces and IPC which are later hard to >>> > develop further and cause latencies. >>> >>> Sure that is the technical best fitting solution for that, we >>> do not want to use separate IPC for the composition approach. >>> > So I'm roughly on the same track as Andreas Pokorny. >>> >>> We too, but unfortunately it is not possible all the time to get >>> the complete user interface included in the compositor itself. We >>> have to guarantee fast startup time to get early video running in 2 >>> seconds. On different systems we can not achieve that with complete >>> HMI-UI is up. >> >> >> Oh, no, I am not suggesting to put user interfaces in the compositor, >> not at all. Only the window management, which means window positioning, >> z-order, etc. For instance, in Weston virtual desktops are implemented >> in the compositor, and not by an external process juggling window lists >> via IPC and setting visibilities and positions. >> >> Proposing just like with weston and desktop-shell, there is the weston >> plugin called desktop-shell.so which does window management, and then >> there is the helper program weston-desktop-shell, that does all the UI >> drawing in a separate process. >> >> Like Jason Ekstrand explained, desktop-shell.so and >> weston-desktop-shell communicate with each other not only using the >> core Wayland protocol, but also using a private (or proprietary) >> protocol extension. >> >> While the basic IPC mechanism is all the same (Wayland), there are >> different parts: extensions. The desktop-shell extension is actually a >> protected one, only weston-desktop-shell as started by desktop-shell.so >> can access it. >> >>> > Have you tried to map your IVI concepts of surface/layer/display to >>> > Wayland wl_surface, wl_subsurface, and wl_output? I don't really >>> > see what kind of interfaces your applications (Wayland clients?) >>> > expect to use. >>> > >>> > When I look at the protocol in ivi-shell.xml, I get the feeling >>> > that: >>> > - Interfaces ivi_layer, ivi_controller_surface, >>> > ivi_controller_layer, ivi_controller_screen, and ivi_controller >>> > should be internal implementation details inside the weston >>> > process, not protocol. Having these as interfaces looks like the X >>> > architecture, where the X server process and window manager process >>> > continuously struggle to keep each other up-to-date, and carefully >>> > try to keep state in sync (and fail), which also makes races and >>> > glitches practically unavoidable. - Interface ivi_client is just a >>> > reinvention of wl_compositor and wl_subcompositor. >>> > - Interface ivi_surface is a reinvention of wl_surface. >>> >>> Let me explain, why this separation occurs. >>> From the ivi perspective we have separated the different applications >>> in ivi-client, ivi-control. >>> >>> ivi-client: Is just a client, like browser, navigation which is >>> embedded in the browser. That belongs to the ivi-surface. These >>> process have really limited access the can just create the surface >>> and that's all. The only important point which the want to know is if >>> the visibility has changed and the size to adapt the rendered >>> content. A navigation should not render the content if it is not >>> visible or assigned to a layer. >> >> >> That is actually very well fitting to what we already have in Wayland. >> Just replace ivi-client with a wl_client (that's not a protocol >> object, it represents a display connection from a client process), and >> ivi-surface with wl_surface+wl_shell_surface. >> >> The limited access is already there: in Weston, clients are free to >> create any number of wl_surfaces. However, no surface is shown >> anywhere on screen, unless it is assigned a suitable role. Examples of >> roles are: "pointer cursor", "top-level window", "drag icon", and >> "popup (menu)". Without a role, the compositor cannot know what to do >> with a surface. >> >> "top-level window" and "popup menu" are shell concepts. If you replace >> wl_shell with your own ivi_shell, you could have roles like: "TV >> output", "rear camera", "speedometer". I'm totally pulling these out of >> my hat, I don't know what actually makes sense for you. >> >> Then, your shell module in weston could handle such surfaces as you >> want. For example, if you get a surface with the role "TV output", the >> shell plugin could show that surface cloned on all entertainment >> monitors. Or whatever. >> >> Or if something signals that now would be a /really/ good time to show >> the rear view camera, then your shell plugin can make it so, if there >> is a surface with such role assigned. >> >> Now, something like a "rear view camera" sounds like a pretty important >> role for a surface, so you may want to make it privileged. You don't >> want any random application to claim to be a rear view camera. In that >> case, you can simply add whatever authentication you need to do in the >> shell plugin. For example, only this one process started directly by >> the shell plugin is allowed to claim the rear view camera role. Anyone >> else trying that will get killed. On the protocol design, such >> protection is pretty irrelevant. All you need is for the shell plugin >> to know who is authorized for what, and it can then enforce the >> restriction, whether the privileged role is requested by a method in a >> otherwise public protocol extension or in its own extension. >> >>> ivi-controller : This is the main application which is controlling >>> the compositor like the UI. That could be the hmi, but it can also >>> some debug applications and application which is triggering a screen >>> shot of surfaces, layer, displays. If you driving a car during the >>> development you want to get some screenshots which are normally >>> triggered by an external application like the log and trace framework. >> >> >> Ok. I am proposing to have window management in the compositor plugin. >> But, you can offer additional debugging interfaces in the protocol. >> >> For example, Weston does not allow clients to control *any* input >> devices. Clients simply cannot feed fake input events to other clients. >> But for testing and debugging, we need a way to fake input from a >> client. The solution: a test plugin. >> >> The test pluging, which is loaded by the compositor only for testing >> (in your case debugging) purposes registers a new protocol extension: >> the test interface. That test interface then allows to do all kinds of >> ugly things from clients. >> >> I'm not sure what you mean by "UI", but we separate controlling windows >> from drawing content, like I explained of the desktop shell. >> >> As a recap, the shell plugin is what enforces and executes policies. >> Clients only indicate intent or purpose, like "This surface is a menu, >> and I want it to pop up right there on my window." A menu is an >> interesting example, since it is more than just a surface role: it >> comes with an input device grab. While the menu is up, input is >> directed to it. But, it is the shell plugin that can decide, that ok, >> you're done now, grab is broken, the menu surface hidden, and the >> client is signalled that something dismissed the menu. Also, clients >> cannot put up a menu at random times, it has to be done as a response >> to an input event. All this prevents many kinds of client misbehaviour. >> >>> > Yes, I see there are some details to may want to control like >>> > surface opacity, that the current Wayland protocols do not support, >>> > but I don't think that replacing everything is a good way to start. >>> > It is also very hard to see how objects from all these interfaces >>> > are created, and how (if?) they associate to any other protocol >>> > objects. >>> >>> On goal is to get setup a complete scenery during the startup of the >>> compositor. Therefore the compositor is able to just initialize the >>> main visible attribute during startup. That implies too that it >>> should be possible, without the final application is running (like >>> navigation). Therefore we need some logical objects like ivi-surface, >>> ivi-layer. Which can be already controlled, whithout a application >>> which holds the content is already running. >> >> >> But if all layer and window management is internal to the compositor >> process, then you don't need any of this control protocol. >> >> Just create some internal data structures, and when a client comes up >> with a surface and requests a particular role, you can plug that >> surface into the slot you prepared in advance. >> >> So, yes, this does imply that the compositor or the shell plugin knows >> the scenegraph, but... isn't that exactly the compositor's purpose, to >> gather the scenegraph and get it on screen? Clients "only" provide >> content. >> >> For example, the desktop-shell protocol extension has special surface >> roles "background" and "panel". When weston-desktop-shell client >> assigns these roles to its surfaces, the compositor (shell plugin) will >> immediately plug them into the correct places on screen. If the shell >> plugin wanted to reserve screen space for them in advance, it could. >> After all, the shell plugin is responsible for arranging everything on >> (and off) screen. >> >>> > Btw. if you need support for surface scaling and cropping, there >>> > have been discussion on the Wayland mailing list to bring a crop & >>> > scale protocol extension to Wayland. It is actually necessary for >>> > efficient video playback etc., so pushing that forward would be >>> > nice. >>> >>> Yeah I see that, and we will consider it. I think we can redesign the >>> approach like source and destination region to the crop and scale >>> instead. >>> >>> > After looking through the two links you gave, the ivi-shell.xml, >>> > and what you have wrote in the emails, I still have no clue what is >>> > the big picture here. >>> > >>> > - What processes are going to use which interfaces? It looks to me >>> > like some interfaces are not meant for all Wayland clients, but >>> > how is it supposed to work? >>> > - What components are in a whole IVI system, from the point of view >>> > of Wayland protocol? What are the responsibilities of each >>> > component and how are these distributed into processes? >>> >>> ivi-control : Is the only process which is allowed to control the >>> compositor itself. But they need some informations about other >>> applications which are running. In the ivi-domain, each graphical >>> application has a unique ID which is defined by the OEM or platform >>> provider. ivi-client : are just other applications like browser or >>> navigation. >> >> >> Ok, so you probably need a protocol interface to tell the compositor >> the unique ID, and perhaps an authentication/authorization interface, >> so that a shell plugin has all the needed information to do the >> controlling. >> >>> > - What does a typical IVI application do in terms of Wayland >>> > protocol? Are you using wl_compositor at all? Or any other >>> > Wayland core interfaces? >>> >>> We were focused on just using the wayland core interface. But we are >>> open to extend this approach. >> >> >> For the record, "Wayland core" is everything in >> <a href="http://cgit.freedesktop.org/wayland/wayland/tree/protocol/wayland.xml">http://cgit.freedesktop.org/wayland/wayland/tree/protocol/wayland.xml</a> >> except wl_shell and wl_shell_surface, which are specific to desktop >> applications. But you are free to pick and choose individual global >> interfaces. >> >>> > The protocol you propose seems to have many references to >>> > "id_native" and "native content", what is all this "native" stuff >>> > about? Or all the integer id's you seem to be sending back and >>> > forth, why can't you use real protocol objects to refer to those? >>> >>> This is a little bit confusing, the initial set of these objects like >>> it is defined here : >>> >>> <a href="http://git.projects.genivi.org/?p=layer_management.git;a=blob;f=wayland-ivi-extension/protocol/ivi-shell.xml;h=6092b13ef31e55f3db338be2e471d0e956be5b9e;hb=ivi-extension">http://git.projects.genivi.org/?p=layer_management.git;a=blob;f=wayland-ivi-extension/protocol/ivi-shell.xml;h=6092b13ef31e55f3db338be2e471d0e956be5b9e;hb=ivi-extension</a> >>> >>> Is focused on using the ivi-surface instead of the native id. The >>> rational behind that is to define unique ids over the complete >>> process space. Therefore the HMI is able to control the different >>> ivi-surface properties. >> >> >> Ok, you are inventing global names for these protocol objects, so you >> could do "window management" in a separate process, right? If you made >> the HMI controller a compositor plugin, these global names would not be >> needed, perhaps? >> >> Perhaps I still do not fully grasp what the HMI controller is. And I >> have more of your emails to read still. >> >> You may have a very good reason to keep the HMI controller separate, >> but the thing I am questioning is, is the interface towards the >> compositor the right one. Should the controller really be responsible >> for window management, or should it just send simple messages like >> "rear view camera on/off" or even "gear: reverse". >> >> Even if the HMI controller was one big block in your architecture >> diagrams, you could split it into a compositor plugin and an external >> process. Then the private Wayland protocol extension they use to >> communicate with each other will be just an implementation detail of >> the whole block, which you might not need to standardize. >> >> There actually exists examples of such a design: most of the >> Wayland-enabled EGL implementations use a private protocol extension to >> communicate between the server and the client. The trick is, both the >> server and the clients load the same libEGL, so the protocol becomes a >> completely hidden implementation detail. >> >>> In the end we are very lucky about any input to get a final best >>> fitting solution for both the ivi aspects and the final >>> implementation for that in the FOSS domain. >> >> >> I am very glad you came forward. :-) >> >> >> Thanks, >> pq > > _______________________________________________ > wayland-devel mailing list > <a href="mailto:wayland-devel@lists.freedesktop.org">wayland-devel@lists.freedesktop.org</a> > <a href="http://lists.freedesktop.org/mailman/listinfo/wayland-devel">http://lists.freedesktop.org/mailman/listinfo/wayland-devel</a>