Multiprocess rendering in wayland - webkitgtk+

Wed Jul 17 19:01:22 PDT 2013

> On Mon, Jul 08, 2013 at 11:13:15PM -0700, yan.wang at linux.intel.com wrote:
>> Hi,
>>   I have implemented Wayland buffer sharing mechanism in WebKit2-efl
>> based
>> on nested client example. Nested client share buffer from one nested
>> client to nesting client which is the Wayland server of nested client
>> and Wayland client of Weston at the same time.
>>   For WebKit2, there may be no only only one buffer which need be
>> shared.
>> So the basic idea is implementing a simple Wayland compositor/server (I
>> called as Mini server.) like Weston on UI process Which could maintain a
>> main loop to accept the link request from Web Process by another Wayland
>> socket (E.g. wayland-10) because Weston use wayland-0 by default.
>
> It would be much better if there's a way to avoid creating the listen
> socket alltogether.  If you look at launch_client() in nested.c, you
> can see how it creates a socketpair and exports one end to the client
> by setting the environment variable WAYLAND_SOCKET to the fd.
> libwayland-client.so supports this by looking for this variable in
> wl_display_connect() and will use the specified fd as the connection
> instead of trying to connect to a listen socket.

I understand your idea of your nest example. But as my understanding, it
will keep 1:1 between nest and nest client. In WebKit, I hope one mini
server could accept the requests from multiply clients because Webkit may
have multiply surfaces shared between UI and Web process. If I am wrong,
please tell me.

>
>>   This simple Wayland server only need implement wl_compositor_interface
>> and wl_surface_interface.
>>   After web process creates a Wayalnd EGL window surface (Wayland hasn't
>> Pixmap like X), we need call eglSwapBuffers twice. In the 1st swapping,
>> surface_attach callback of wl_surface_interface will give you a
>> wl_buffer pointer from graphics driver. You could use this wl_buffer to
>> create EGL image / texture on UI process which includes the web content
>> drawn by Web process. The 2nd swapping will swap your previous wl_buffer
>> to back buffer of EGL window surface in graphics driver for drawing on
>> Web process. Just ignore wl_buffer of 2nd swapping.
>
> I don't understand the reason for calling eglSwapBuffers twice.  Why
> is that necessary?

In my previous prototype, wl_egl_window is used instead of X pixmap to
create EGL surface. EGL Surface based on wl_egl_window has 2 wl_buffer (or
3 wl_buffer) for swapping. And EGL surface on X pixmap only has a
wl_buffer. So I want to use back buffer of EGL Window Surface as my
"pixmap". The first swapping could receive the wl_buffer pointer in
surface attach callback. And the 2nd swapping could make previous received
wl_buffer is switched to back buffer for drawing on Web process side. So I
just use one wl_buffer (back buffer) for sharing between UI and Web
process.

Now Kalyan has changed the mechanism in WebKit upstream (shared-surface
branch) of tzien.org. He uses both 2 wl_buffer fully and avoid use
multiply platform surfaces like X in WebGL context
(GraphicsContext3DOffscreen). So This issue isn't exist.

Yan Wang

>
> Kristian
>
>>   About keeping the pair of wl_buffer/EGL window surface, we could have
>> several method. One is add a event into wl_surface_interface protocol.
>> It si simple. Another method to use id of compositorCreateSurafce
>> callback of wl_compositor_surface.
>>   In general, the idea use wl_egl_window instead of X pixmap on Web
>> process, and use EGL image/texture from wl_buffer to do compositing on
>> UI process.
>>   Hope the above idea useful for you.
>>   Thanks.
>>
>> Yan Wang
>>
>>
>> > On Mon, Jul 8, 2013 at 3:40 PM, Iago Toral <itoral at igalia.com> wrote:
>> >> El 2013-07-08 08:38, Jonas Ã
dahl escribiÃ³:
>> >>
>> >>> On Mon, Jul 8, 2013 at 8:05 AM, Iago Toral <itoral at igalia.com>
>> wrote:
>> >>>>
>> >>>> Hi,
>> >>>>
>> >>>> I am working on porting WebKitGTK+ to Wayland and we are having
>> some
>> >>>> difficulties figuring out the proper way to deal with the
>> multiprocess
>> >>>> architecture introduced with WebKit2.
>> >>>>
>> >>>> In WebKit2 we have two processes that are responsible for rendering
>> >>>> the
>> >>>> contents of a webpage. The WebProcess takes care of parsing HTML,
>> >>>> identifying the various layers that are part of that HTML (that
>> will
>> >>>> be
>> >>>> rendered separately) and the composition of all these layers to
>> create
>> >>>> the
>> >>>> final view of the page. This composition stage is done with OpenGL.
>> >>>> Once
>> >>>> the
>> >>>> composition is done, the other process (UIProcess) needs a way to
>> >>>> access
>> >>>> the
>> >>>> results of the composition and paint them on the screen.
>> >>>>
>> >>>> In X11, this is achieved by having the WebProcess render the
>> >>>> composition
>> >>>> results to an offscreen XWindow and sharing the XWindow ID between
>> the
>> >>>> two
>> >>>> processes. XComposite is used to redirect the XWindow to a pixmap.
>> The
>> >>>> pixmap is painted in the UIProcess.
>> >>>>
>> >>>> As far as we know, there is no API in Wayland to allow two
>> different
>> >>>> processes to share a surface, so we are not sure if the
>> architecture I
>> >>>> describe above is currently supported in Wayland.
>> >>>>
>> >>>> So I guess my questions are:
>> >>>> - Is there a way to share a surface between two processes?
>> >>>> - If not, is there a way to implement this architecture in Wayland
>> as
>> >>>> it
>> >>>> is
>> >>>> now?
>> >>>> - Would it be possible/interesting to add surface sharing API to
>> >>>> Wayland
>> >>>> so
>> >>>> that it supports this type of architectures naturally?
>> >>>
>> >>>
>> >>> I proposed an extension[0] for solving this a while back, but since
>> >>> then as far as I know the general consensus has been to use nested
>> >>> compositing[1] for sharing surfaces between processes. The nested
>> >>> compositing is possible now, but if I remember correctly, it will
>> >>> require an extra draw, as there is no Wayland EGL API for directly
>> >>> providing buffers from a nested client to a surface of the host
>> >>> client. Regarding the mentioned extension, I had a hacked up
>> >>> proof-of-concept working, but have not continued working on it
>> >>> considering that it nested compositing and added EGL API is supposed
>> >>> to be the way forward. If I have understood the situation wrong, I'd
>> >>> be happy to continue with the previously proposed protocol
>> extension.
>> >>>
>> >>> [0]
>> >>>
>> >>> http://lists.freedesktop.org/archives/wayland-devel/2013-March/008093.html
>> >>> [1] http://cgit.freedesktop.org/wayland/weston/tree/clients/nested.c
>> >>>
>> >>> Jonas
>> >>
>> >>
>> >> Thanks Jonas, that is useful, I will look into it. The need for an
>> extra
>> >> draw is suboptimal though, do I understand correctly that this extra
>> >> Wayland
>> >> EGL API you mention is intended to fix this?
>> >
>> > Yes, at least that is my understanding of it. I don't think I've seen
>> > any patches for it though, except this[0] which has some similarities.
>> >
>> > Jonas
>> >
>> > [0]
>> > http://lists.freedesktop.org/archives/wayland-devel/2013-March/007746.html
>> >
>> >>
>> >> Iago
>> > _______________________________________________
>> > wayland-devel mailing list
>> > wayland-devel at lists.freedesktop.org
>> > http://lists.freedesktop.org/mailman/listinfo/wayland-devel
>> >
>> _______________________________________________
>> wayland-devel mailing list
>> wayland-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
>