[RFC] wl_surface video protocol extension

[Frederic Plourde]

Hi everybody

We're having needs for a streaming_video-capable wl_surface. An extended 
surface that could "queue buffers up" along with presentation timestamps 
in the compositor so that videosink clients (like gstreamer's wayland 
videosink) could more effectively and precisely synchronize audio and 
video on specific time cues.

I'd like your comments/opinion about this first draft of the protocol. 
Protocol and interfaces names, language, set of requests, etc...

The idea for now is to let the clients queue buffers in advance and 
clear the queue abruptely for cases where they'd need, e.g. to "pause" 
the streaming.We're still having a whole bunch of implementation-wise 
questions, but that's a start. It'd be awesome if I had your feedback on 
this protocol.

Thanks !
Frederic Plourde




<?xml version="1.0" encoding="UTF-8"?>
<protocol name="videosurface">

   <copyright>
     Copyright © 2012-2013 Collabora, Ltd.

     Permission to use, copy, modify, distribute, and sell this
     software and its documentation for any purpose is hereby granted
     without fee, provided that the above copyright notice appear in
     all copies and that both that copyright notice and this permission
     notice appear in supporting documentation, and that the name of
     the copyright holders not be used in advertising or publicity
     pertaining to distribution of the software without specific,
     written prior permission.  The copyright holders make no
     representations about the suitability of this software for any
     purpose.  It is provided "as is" without express or implied
     warranty.

     THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS
     SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
     FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY
     SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
     WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
     AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
     ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
     THIS SOFTWARE.
   </copyright>

   <interface name="wl_videocompositor" version="1">
     <description summary="video-surface compositing">
       The global interface exposing video-surface compositing capabilities.

       The aim of video-surfaces is to support streaming video coming from
       videosinks that typically need to queue video buffers with 
presentation
       timestamps in the compositor in order to accurately synchronize video
       and audio streams.
     </description>

     <request name="destroy" type="destructor">
       <description summary="unbind from the videocompositor interface">
         Tell the server that the client will not be using this
         protocol object anymore. This does not affect any other
         objects, wl_videosurface objects included.
       </description>
     </request>

   <enum name="error">
       <entry name="video_surface_exists" value="0"
              summary="the surface already has a video-surface object 
associated"/>
     </enum>

     <request name="get_videosurface">
       <description summary="promote a surface to a video-surface">
         Create and attach a video-surface interface for the given
         surface. This effectively turns a plain wl_surface into a
         specialized video-surface with video-streaming capabilities
         via the use of buffer queues. If the given wl_surface already has
         a wl_video_surface object associated, the video_surface_exists
         protocol error is raised.
       </description>

       <arg name="id" type="new_id" interface="wl_videosurface"
            summary="the new videosurface object id"/>
       <arg name="surface" type="object" interface="wl_surface"
            summary="the surface to be turned into a video-surface"/>
     </request>
   </interface>

   <interface name="wl_videosurface" version="1">
     <description summary="video-surface interface to a wl_surface">
       An additional interface to a wl_surface object, which has been
       promoted to a video-surface. A video-surface contains a buffer
       queue and exposes requests that allow one to add a timestamped
       wl_buffer into the video-surface buffer queue.

       Video-surfaces do not declare any state in addition to the ones
       plain surfaces already use by default). Thus, once a plain sur-
       face has been created and promoted to a video-surface, buffer
       queuing happens through "queue_buffer requests". This means that
       queuing up buffers does *not* require any surface.attach nor
       surface.commit. Moreover, such an .attach or .commit will
       clear the buffer queue in order to exclusively attach the newly
       provided buffer via .attach. Hence, surface.attach and surface.
       commit should *not* be used when specifying streaming buffer
       content of a video-surface, as they are only relevant for single-
       buffer plain surfaces.

       At every repaint cycle, the compositor will look for past-
       timestamped buffers in the queue and if any, will render the
       most recent one of them. Older buffers will be removed from the
       list at that point and destroyed (an BUFFER_RELEASE event will
       be fired for every removed one). At every repaint cycle, future-
       timestamped (or posterior) buffers will be kept untouched.

       while enqueuing video-surface buffers, it's not necessary to
       specifying any 'damage' state because enqueued buffers will
       automatically be associated with fullscreen damage.
     </description>

     <request name="destroy" type="destructor">
       <description summary="remove video-surface interface">
         The video-surface interface is removed from the wl_surface object
         that was turned into a video-surface with
         wl_videocompositor.get_videosurface request.
       </description>
     </request>

     <request name="queue_buffer">
       <description summary="enqueue a buffer into the surface's queue">
         Queue a buffer along with its timestamp into the video-surface
         internal buffer queue for later/synchronized presentation to 
display
         hardware.
       </description>

       <arg name="buffer" type="object" interface="wl_buffer"
            summary="the buffer to queue"/>
       <arg name="timestamp" type="uint"
            summary="the buffer's presentation timestamp"/>
     </request>

     <request name="clear_buffer_queue">
       <description summary="clear the surface's buffer queue">
         Clear the surface's queue from all its buffers. None of the removed
         buffers will be presented at given times and associated timestamps
         are cleared from wl_buffer implementations.
       </description>
     </request>

   </interface>

   </interface>
</protocol>