[RFC v2] protocol: Introduce foreign surface protocol

Sat Mar 23 08:03:33 PDT 2013

On Fri, Mar 22, 2013 at 5:06 PM, Jason Ekstrand <jason at jlekstrand.net> wrote:
> Hi Jonas,

Hi Jason,

> I'm really starting to like the look of this.   However there are
> still a few sticky spots. First: How do you plan to handle resizing in
> a sane way? Do you plan to do that entirely through the external IPC
> mechanism or have some sort of configure event that gets passed via
> wayland?  Also, there's a potential treading issue: see below

I think resizing can work in the same way as for sub-surfaces, as in
its up to the clients to coordinate resizing. During resize, they
still would need to synchronize frames with each other so that no
frame will have size mismatches.

>
> On Fri, Mar 22, 2013 at 2:49 AM, Jonas Ådahl <jadahl at gmail.com> wrote:
>> A foreign surface is a surface created given a handle associated with a
>> previously exported surface. A handle is a randomly generated unique
>> identifier that on the server can associate a handle with a surface.
>>
>> The exported surface is a surface created as normal but exported via the
>> wl_foreign_surface_manager to be used to create a foreign surface on
>> another client. The client of the exported surface is responsible for
>> attaching buffers, setting input regions etc but the client of the
>> foreign surface is responsible for mapping it. The foreign surface is
>> created by sending the wl_foreign_surface_manager.create_foreign_surface
>> request with the handle shared via some IPC.
>>
>> The surface created by create_foreign_surface can only be mapped as a
>> sub-surface via the wl_subcompositor interface.
>>
>> A possible use case is:
>>
>> There are two clients (A and B) that can communicate via some IPC
>> mechanism where A is the main process and B is a render process. B
>> renders to a surface and A decides its visibility and position. Below
>> follows an example how foreign surfaces can be used.
>>
>> 1. Both A and B connects do the server
>> 5. A creates a main surface
>> 2. B creates a render surface
>> 3. B exports the render surface and retrieves a handle
>> 4. B sends the surface handle via IPC to A
>> 6. A receives the surface handle via IPC from B
>> 7. A creates a foreign wl_surface via wl_foreign_surface_manager
>> 8. A creates a wl_subsurface via the wl_subcompositor
>>
>> B can now render frames to its exported surface, and it will be mapped
>> and composited according to A's choice of positioning.
>>
>> Signed-off-by: Jonas Ådahl <jadahl at gmail.com>
>> ---
>>
>> Hi,
>>
>> This is a follow-up of the "[RFC] Introduce logical surface protocol" [0].
>>
>> Some changes since version 1 are:
>>  - renamed to foreign surfaces,
>>  - export new surface object instead of acquire/release handle,
>>  - create only wl_surface via the new interface,
>>  - create the wl_subsurface interface directly via the wl_subcompositor,
>>
>> Some unfinished parts are how to route with input and how to deal with
>> involuntary destruction of exported surfaces (crashes).
>>
>> Input routing would depend on how sub-surfaces deal with input routing. Not
>> sure there has been any conclusions regarding that yet.
>>
>> For handling lost surfaces I see at least two possible ways to have some
>> damage control, and that is to either keep around a buffer of a destroyed
>> surface, or have the foreign surface client attach a "backup" buffer for
>> displaying when the other surface was lost. The backup buffer approach
>> is quite similar to just having another surface behind though, so might
>> not be a very good idea.
>>
>>
>> Jonas
>>
>>  [0] http://lists.freedesktop.org/archives/wayland-devel/2013-March/007986.html
>>
>>
>>
>>  protocol/foreign-surface.xml |  122 ++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 122 insertions(+)
>>  create mode 100644 protocol/foreign-surface.xml
>>
>> diff --git a/protocol/foreign-surface.xml b/protocol/foreign-surface.xml
>> new file mode 100644
>> index 0000000..442f0f1
>> --- /dev/null
>> +++ b/protocol/foreign-surface.xml
>> @@ -0,0 +1,122 @@
>> +<protocol name="foreign_surface">
>> +
>> +  <copyright>
>> +    Copyright © 2013 Jonas Ådahl
>> +
>> +    Permission to use, copy, modify, distribute, and sell this
>> +    software and its documentation for any purpose is hereby granted
>> +    without fee, provided that the above copyright notice appear in
>> +    all copies and that both that copyright notice and this permission
>> +    notice appear in supporting documentation, and that the name of
>> +    the copyright holders not be used in advertising or publicity
>> +    pertaining to distribution of the software without specific,
>> +    written prior permission.  The copyright holders make no
>> +    representations about the suitability of this software for any
>> +    purpose.  It is provided "as is" without express or implied
>> +    warranty.
>> +
>> +    THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS
>> +    SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
>> +    FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY
>> +    SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
>> +    WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
>> +    AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
>> +    ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
>> +    THIS SOFTWARE.
>> +  </copyright>
>> +
>> +  <interface name="wl_foreign_surface_manager" version="1">
>> +    <description summary="foreign surface manager">
>> +      A foreign surface is a surface mapped by one client, but rendered
>> +      by another. By exporting a surface the exporter will acquire a
>> +      global surface handle that can be shared with another client. The
>> +      other client can use handle to create a foreign "wl_surface" which
>> +      then can be used to create a "wl_subsurface" interface object. A
>> +      foreign surface can only be mapped as a sub-surface.
>> +
>> +      When having mapped a foreign surface, buffers attached to the
>> +      corresponding exported surface will be composited as if they were
>> +      mapped as the foreign surface.
>> +    </description>
>> +
>> +    <enum name="error">
>> +      <entry name="no_surface" value="0"
>> +            summary="no suitable surface found"/>
>> +    </enum>
>> +
>> +    <request name="create_foreign_surface">
>> +      <description summary="create a foreign surface">
>> +       Given a surface handle of an "wl_exported_surface" create a new
>> +       foreign surface associated with the exported surface. The new
>> +       surface can only be used as a sub-surface by creating a
>> +       "wl_sub_surface" interface object via "wl_subcompositor".
>> +
>> +       If no suitable surface found an error will be raised.
>> +      </description>
>> +
>> +      <arg name="handle" type="string"/>
>> +      <arg name="id" type="new_id" interface="wl_surface"/>
>> +    </request>
>> +
>> +    <request name="export_surface">
>> +      <description summary="export a surface">
>> +       Export a surface to be used to create a foreign surface on some
>> +       client, for example a client other than the one doing the request
>> +       This turns the plain surface into an exported surface, and it
>> +       cannot be re-mapped until the exported surface interface object
>> +       is destroyed.
>> +      </description>
>> +
>> +      <arg name="surface" type="object" interface="wl_surface"/>
>> +      <arg name="id" type="new_id" interface="wl_exported_surface"/>
>> +    </request>
>> +
>> +    <event name="foreign_surface_lost">
>> +      <description summary="foreign surface lost">
>> +       The exported surface associated with the foreign surface
>> +       "surface" has been destroyed. This means that "surface" and
>> +       potentially an associated "wl_subsurface" are now invalid and
>> +       should be destroyed.
>> +      </description>
>> +
>> +      <arg name="surface" type="object" interface="wl_surface"/>
>> +    </event>
>
> If the client exporting surfaces is running in multiple threads, this
> event may an issue.  It's not necessarily fatal, so clients should be
> able to handle it nicely.  However, unless you plan to have each
> thread bind to the foreign surface manager separately (probably not a
> good idea), this isn't going to thread well.
>
> One possible way to solve this is to add a wl_foreign_surface
> interface with a create_surface event and a surface_lost event.
>

I think multi threaded exporting client won't be an issue for the lost
event since it's sent to the client that created the foreign surface.
A foreign surface using client could potentially use some threading
model to avoid problems with a centralized manager, but I think what
you are proposing here makes sense, as it doesn't enforce some
threading model, and it better corresponds to what the event
represent. The wl_foreign_surface lifetime could be bound to the one
of the wl_surface similar to as how wl_shell_surface works now.

>> +  </interface>
>> +
>> +  <interface name="wl_exported_surface" version="1">
>> +    <description summary="exported surface">
>> +      An additional interface object for an exported wl_surface. An
>> +      exported surface works like a normal surface, except for that it
>> +      gets mapped via a foreign surface created by
>> +      wl_foreign_surface_manager.create_foreign_surface.
>> +
>> +      The task of the exported surface is to attach buffers, set input,
>> +      opaque and damaged regions, and commit.
>> +    </description>
>> +
>> +    <request name="destroy">
>> +      <description summary="release exported surface">
>> +       Destroys the exported surface interface object and turns the
>> +       surface that was exported to an un-mapped plain surface. If some
>> +       foreign surface is associated with this surface an
>> +       "foreign_surface_lost" event is emitted to the foreign surface
>> +       manager with the foreign surface specified.
>> +      </description>
>> +    </request>
>> +
>> +    <event name="handle">
>> +      <description summary="surface handle">
>> +       This event will be sent immediately after the
>> +       "wl_exported_surface" interface object was created on the server.
>> +
>> +       The handle is a randomly generated unique identifier that allows
>> +       the server to identify the "wl_surface" used to create this
>> +       "wl_exported_surface" and create a associated
>> +       "wl_foreign_surface".
>> +      </description>
>> +
>> +      <arg name="handle" type="string"/>
>> +    </event>
>> +  </interface>
>> +
>> +</protocol>
>> --
>> 1.7.10.4
>>
>> _______________________________________________
>> wayland-devel mailing list
>> wayland-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
>
> Thanks,
> --Jason Ekstrand

Again, thanks for your input,

Jonas