[RFC] wl_surface video protocol extension
frederic.plourde at collabora.co.uk
Wed Oct 16 17:27:44 CEST 2013
We're having needs for a streaming_video-capable wl_surface. An extended
surface that could "queue buffers up" along with presentation timestamps
in the compositor so that videosink clients (like gstreamer's wayland
videosink) could more effectively and precisely synchronize audio and
video on specific time cues.
I'd like your comments/opinion about this first draft of the protocol.
Protocol and interfaces names, language, set of requests, etc...
The idea for now is to let the clients queue buffers in advance and
clear the queue abruptely for cases where they'd need, e.g. to "pause"
the streaming.We're still having a whole bunch of implementation-wise
questions, but that's a start. It'd be awesome if I had your feedback on
<?xml version="1.0" encoding="UTF-8"?>
Copyright © 2012-2013 Collabora, Ltd.
Permission to use, copy, modify, distribute, and sell this
software and its documentation for any purpose is hereby granted
without fee, provided that the above copyright notice appear in
all copies and that both that copyright notice and this permission
notice appear in supporting documentation, and that the name of
the copyright holders not be used in advertising or publicity
pertaining to distribution of the software without specific,
written prior permission. The copyright holders make no
representations about the suitability of this software for any
purpose. It is provided "as is" without express or implied
THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS
SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY
SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
<interface name="wl_videocompositor" version="1">
<description summary="video-surface compositing">
The global interface exposing video-surface compositing capabilities.
The aim of video-surfaces is to support streaming video coming from
videosinks that typically need to queue video buffers with
timestamps in the compositor in order to accurately synchronize video
and audio streams.
<request name="destroy" type="destructor">
<description summary="unbind from the videocompositor interface">
Tell the server that the client will not be using this
protocol object anymore. This does not affect any other
objects, wl_videosurface objects included.
<entry name="video_surface_exists" value="0"
summary="the surface already has a video-surface object
<description summary="promote a surface to a video-surface">
Create and attach a video-surface interface for the given
surface. This effectively turns a plain wl_surface into a
specialized video-surface with video-streaming capabilities
via the use of buffer queues. If the given wl_surface already has
a wl_video_surface object associated, the video_surface_exists
protocol error is raised.
<arg name="id" type="new_id" interface="wl_videosurface"
summary="the new videosurface object id"/>
<arg name="surface" type="object" interface="wl_surface"
summary="the surface to be turned into a video-surface"/>
<interface name="wl_videosurface" version="1">
<description summary="video-surface interface to a wl_surface">
An additional interface to a wl_surface object, which has been
promoted to a video-surface. A video-surface contains a buffer
queue and exposes requests that allow one to add a timestamped
wl_buffer into the video-surface buffer queue.
Video-surfaces do not declare any state in addition to the ones
plain surfaces already use by default). Thus, once a plain sur-
face has been created and promoted to a video-surface, buffer
queuing happens through "queue_buffer requests". This means that
queuing up buffers does *not* require any surface.attach nor
surface.commit. Moreover, such an .attach or .commit will
clear the buffer queue in order to exclusively attach the newly
provided buffer via .attach. Hence, surface.attach and surface.
commit should *not* be used when specifying streaming buffer
content of a video-surface, as they are only relevant for single-
buffer plain surfaces.
At every repaint cycle, the compositor will look for past-
timestamped buffers in the queue and if any, will render the
most recent one of them. Older buffers will be removed from the
list at that point and destroyed (an BUFFER_RELEASE event will
be fired for every removed one). At every repaint cycle, future-
timestamped (or posterior) buffers will be kept untouched.
while enqueuing video-surface buffers, it's not necessary to
specifying any 'damage' state because enqueued buffers will
automatically be associated with fullscreen damage.
<request name="destroy" type="destructor">
<description summary="remove video-surface interface">
The video-surface interface is removed from the wl_surface object
that was turned into a video-surface with
<description summary="enqueue a buffer into the surface's queue">
Queue a buffer along with its timestamp into the video-surface
internal buffer queue for later/synchronized presentation to
<arg name="buffer" type="object" interface="wl_buffer"
summary="the buffer to queue"/>
<arg name="timestamp" type="uint"
summary="the buffer's presentation timestamp"/>
<description summary="clear the surface's buffer queue">
Clear the surface's queue from all its buffers. None of the removed
buffers will be presented at given times and associated timestamps
are cleared from wl_buffer implementations.
More information about the wayland-devel