<html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <div class="moz-cite-prefix">Hi, I have gathered comments and suggestions from colleagues and wayland-devel reviewers and here is RFC v.2 of our buffer queue + presentation feedback protocol extension. Notice the following changes : ----------------------------------------- - it's changed name to make it more general. (comments/ideas on the protocol and interfaces names are welcome). We believe this wl_surface extension really is down to adding a timed buffer queue, hence the wl_buffer_queue interface name. - it's been extended to include presentation time feedback (wl_presentation_time) - Extensive comments were added. please, see the code below: <tt><?xml version="1.0" encoding="UTF-8"?></tt><tt> </tt><tt><protocol name="presentation_timing"></tt><tt> </tt><tt> </tt><tt> <copyright></tt><tt> </tt><tt> Copyright © 2012-2013 Collabora, Ltd.</tt><tt> </tt><tt> </tt><tt> Permission to use, copy, modify, distribute, and sell this</tt><tt> </tt><tt> software and its documentation for any purpose is hereby granted</tt><tt> </tt><tt> without fee, provided that the above copyright notice appear in</tt><tt> </tt><tt> all copies and that both that copyright notice and this permission</tt><tt> </tt><tt> notice appear in supporting documentation, and that the name of</tt><tt> </tt><tt> the copyright holders not be used in advertising or publicity</tt><tt> </tt><tt> pertaining to distribution of the software without specific,</tt><tt> </tt><tt> written prior permission. The copyright holders make no</tt><tt> </tt><tt> representations about the suitability of this software for any</tt><tt> </tt><tt> purpose. It is provided "as is" without express or implied</tt><tt> </tt><tt> warranty.</tt><tt> </tt><tt> </tt><tt> THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS</tt><tt> </tt><tt> SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND</tt><tt> </tt><tt> FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY</tt><tt> </tt><tt> SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES</tt><tt> </tt><tt> WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN</tt><tt> </tt><tt> AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,</tt><tt> </tt><tt> ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF</tt><tt> </tt><tt> THIS SOFTWARE.</tt><tt> </tt><tt> </copyright></tt><tt> </tt><tt> </tt><tt> <interface name="wl_presentation" version="1"></tt><tt> </tt><tt> <description summary="buffer queues with presentation timing information/</tt><tt> </tt><tt> for surfaces"></tt><tt> </tt><tt> The global factory interface exposing timestamped buffer queuing composi-</tt><tt> </tt><tt> ting capabilities.</tt><tt> </tt><tt> </tt><tt> The aim of presentation timing is to support streaming video generally</tt><tt> </tt><tt> coming from videosink clients that typically need to queue video buffers</tt><tt> </tt><tt> with presentation timestamps in order to accurately synchronize video </tt><tt> </tt><tt> and audio streams.</tt><tt> </tt><tt> </tt><tt> This global interface allows for the creation of timestamped buffer </tt><tt> </tt><tt> queues for wl_surface objects.</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> </tt><tt> <request name="destroy" type="destructor"></tt><tt> </tt><tt> <description summary="unbind from the wl_presentation interface"></tt><tt> </tt><tt> Tell the server that the client will not be using this</tt><tt> </tt><tt> protocol object anymore. This does not affect any other</tt><tt> </tt><tt> objects, wl_buffer_queue objects included.</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> </request></tt><tt> </tt><tt> </tt><tt> <enum name="error"></tt><tt> </tt><tt> <entry name="buffer_queue_exists" value="0"</tt><tt> </tt><tt> summary="the surface already has a buffer_queue object/</tt><tt> </tt><tt> associated to it"/></tt><tt> </tt><tt> </enum></tt><tt> </tt><tt> </tt><tt> <event name="clock_id", type="uint"></tt><tt> </tt><tt> <description summary="clock ID to use"></tt><tt> </tt><tt> Tell the client which clock ID is the compositor going to use for time-</tt><tt> </tt><tt> stamps and presentation feedback.</tt><tt> </tt><tt> </tt><tt> Compositor sends that event once to a client right after it binds to </tt><tt> </tt><tt> the global interface.</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> </event></tt><tt> </tt><tt> </tt><tt> <request name="create_queue"></tt><tt> </tt><tt> <description summary="add buffer queueing capabilities to a wl_surface"></tt><tt> </tt><tt> Create and attach a wl_buffer_queue interface to the given wl_sur-</tt><tt> </tt><tt> face. This effectively adds buffer queueing and presentation timing</tt><tt> </tt><tt> feedback capabilities to the surface through the use of buffer queues</tt><tt> </tt><tt> and presentation callbacks.</tt><tt> </tt><tt> </tt><tt> If the given wl_surface already has a wl_buffer_queue object associated</tt><tt> </tt><tt> to it, the buffer_queue_exists protocol error is raised.</tt><tt> </tt><tt> </tt><tt> Here, clock_id is needed to negociate time domain with the compositor</tt><tt> </tt><tt> as a common basis when specifing timestamps and presentation callbacks.</tt><tt> </tt><tt> clock_id is an implementation-specific integer representing the clock</tt><tt> </tt><tt> identification.</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> </tt><tt> <arg name="id" type="new_id" interface="wl_buffer_queue"</tt><tt> </tt><tt> summary="the new buffer_queue object id"/></tt><tt> </tt><tt> <arg name="surface" type="object" interface="wl_surface"</tt><tt> </tt><tt> summary="the surface to be turned intio a buffer_queue surface"/></tt><tt> </tt><tt> <arg name="clock_id" type="uint" summary="clock ID to be used for/</tt><tt> </tt><tt> timestamps"></tt><tt> </tt><tt> </request></tt><tt> </tt><tt> </interface></tt><tt> </tt><tt> </tt><tt> <interface name="wl_buffer_queue" version="1"></tt><tt> </tt><tt> <description summary="buffer queue interface to a wl_surface"></tt><tt> </tt><tt> An additional interface to a wl_surface object to enable buffer queueing</tt><tt> </tt><tt> capabilities. A buffer_queue-enabled surface contains a buffer queue and</tt><tt> </tt><tt> exposes protocol that allows for queuing timestamped wl_buffer objects</tt><tt> </tt><tt> into the queue and requesting presentation time feedback.</tt><tt> </tt><tt> </tt><tt> Buffer_queue-enabled surfaces do not declare any state in addition to the</tt><tt> </tt><tt> ones wl_surfaces already use by default. Thus, once a plain wl_surface has</tt><tt> </tt><tt> been turned into a buffer_queue-enabled surface, buffer queuing hap-</tt><tt> </tt><tt> pens through the "queue" request. This means that queuing up</tt><tt> </tt><tt> buffers does *not* require any surface.attach nor surface.commit. More-</tt><tt> </tt><tt> over, such an attach + commit sequence will clear the buffer queue in</tt><tt> </tt><tt> order to exclusively consider the newly attached buffer. Hence,</tt><tt> </tt><tt> surface.attach and surface.commit should *not* be used when specifying</tt><tt> </tt><tt> streaming surface content via buffer_queue. Also, queuing up a wl_buffer</tt><tt> </tt><tt> makes it "reserved" in the compositor just like attach + commit does.</tt><tt> </tt><tt> </tt><tt> Besides, it is perfectly acceptable to commit (without attach) some</tt><tt> </tt><tt> states on a buffer_queue-enabled surface, as it will not clear the buffer</tt><tt> </tt><tt> queue nor interfere with any presentation timing mechanism brought by</tt><tt> </tt><tt> the presentation_timing interfaces.</tt><tt> </tt><tt> </tt><tt> The compositor will attempt to repaint so that each queued buffer gets</tt><tt> </tt><tt> presented at the requested target time. However, this may not always be</tt><tt> </tt><tt> possible e.g. when a requested presentation time cannot be met accurately</tt><tt> </tt><tt> or when the target time has completely passed already and been replaced by</tt><tt> </tt><tt> another buffer by the time the compositor can repaint again. Therefore, </tt><tt> </tt><tt> At every repaint cycle, enqueued buffers with past-timestamps, if any,</tt><tt> </tt><tt> will be considered and the compositor will present the most recent one</tt><tt> </tt><tt> among them. At that point, buffers with older timestamps will be removed</tt><tt> </tt><tt> from the queue and released (a BUFFER_RELEASE event will be fired for</tt><tt> </tt><tt> every removed one). Also, at every repaint cycle, future-timestamped</tt><tt> </tt><tt> buffers will be kept untouched in the queue.</tt><tt> </tt><tt> </tt><tt> When applying a buffer from the queue, the compositor implies full</tt><tt> </tt><tt> surface damage. If the client manages to attach + damage + commit in the</tt><tt> </tt><tt> meantime, then the damage is what it explicitely specified (and the</tt><tt> </tt><tt> buffer_queue gets cleared)</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> </tt><tt> <request name="destroy" type="destructor"></tt><tt> </tt><tt> <description summary="remove buffer_queue interface"></tt><tt> </tt><tt> The buffer_queue interface is removed from the buffer_queue-enabled</tt><tt> </tt><tt> surface.</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> </request></tt><tt> </tt><tt> </tt><tt> <request name="queue"></tt><tt> </tt><tt> <description summary="enqueue a buffer into the surface's buffer queue"></tt><tt> </tt><tt> Queue a buffer along with its timestamp into the surface's internal</tt><tt> </tt><tt> buffer queue. This timestamp is the intended presentation time. While</tt><tt> </tt><tt> clients generally want to queue posterior timestamps (future), queueing</tt><tt> </tt><tt> up anterior timestamps (in the past) is allowed and does not raise any</tt><tt> </tt><tt> protocol error.</tt><tt> </tt><tt> </tt><tt> The speficied timestamp is two-part. tv_sec is the number of seconds</tt><tt> </tt><tt> and tv_nsec defines the number of nanoseconds spent after tv_sec.</tt><tt> </tt><tt> </tt><tt> Calling queue() will create a wl_presentation_time object that is ex-</tt><tt> </tt><tt> clusively associated with the provided wl_buffer lifetime in the queue</tt><tt> </tt><tt> at the moment the object was created. This means that presentation</tt><tt> </tt><tt> feedback concerning this wl_buffer will be provided through wl_presen-</tt><tt> </tt><tt> tation_time events, but only once. As soon as the buffer is presented</tt><tt> </tt><tt> and removed from the queue by the compositor, it's internal link to</tt><tt> </tt><tt> the wl_presentation_time object is broken and associated presentation</tt><tt> </tt><tt> events will never be called again. So a client, right after having recei-</tt><tt> </tt><tt> ved a wl_presentation_time event will generally destroy that object.</tt><tt> </tt><tt> On the other hand, if a client does not want feedback, it may destroy</tt><tt> </tt><tt> presentation_time object right after queuing a buffer.</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> </tt><tt> <arg name="buffer" type="object" interface="wl_buffer"</tt><tt> </tt><tt> summary="the buffer to queue"/></tt><tt> </tt><tt> <arg name="id" type="new_id" interface="wl_presentation_time"</tt><tt> </tt><tt> summary="the new presentation_time object id"/></tt><tt> </tt><tt> <arg name="tv_sec" type="uint"</tt><tt> </tt><tt> summary="seconds value of the buffer target timestamp"/></tt><tt> </tt><tt> <arg name="tv_nsec" type="uint"</tt><tt> </tt><tt> summary="nanoseconds value of the buffer target timestamp"/></tt><tt> </tt><tt> </request></tt><tt> </tt><tt> </tt><tt> <request name="clear"></tt><tt> </tt><tt> <description summary="clear the surface's buffer queue"></tt><tt> </tt><tt> Clear the surface's buffer queue. All buffers are removed from the</tt><tt> </tt><tt> queue and released as appropriate. The requested presentation times are</tt><tt> </tt><tt> discarded. The surface retains the contents it currently has, at the</tt><tt> </tt><tt> time the compositor processes this request. The compositor may not</tt><tt> </tt><tt> release the buffer(s) currently being displayed if it is needed for</tt><tt> </tt><tt> repainting or scanout, as usual.</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> </request></tt><tt> </tt><tt> </interface></tt><tt> </tt><tt> </tt><tt> <interface name="wl_presentation_time" version="1"></tt><tt> </tt><tt> <description summary="presentation time feedback event"></tt><tt> </tt><tt> The wl_presentation_time object encapsulates presentation time events</tt><tt> </tt><tt> sent as feedback by the compositor to a client that previously queued</tt><tt> </tt><tt> a wl_buffer. wl_presentation_time objects are created and returned when</tt><tt> </tt><tt> a client enqueues a buffer through the wl_buffer_queue.queue() request.</tt><tt> </tt><tt> </tt><tt> Note that presentation time only tells client when the compositor presen-</tt><tt> </tt><tt> ted the buffer to display hardware, not when the buffer was turned into</tt><tt> </tt><tt> light (actually displayed on screen) by this hardware. As there could</tt><tt> </tt><tt> be anything displaying those buffers, from very fast, low-latency</tt><tt> </tt><tt> computer monitors to slow, hi-latency HDMI TV screens, it is the client's</tt><tt> </tt><tt> responsibility to make sure it knows what display hardware is currently</tt><tt> </tt><tt> connected and what is its latency.</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> </tt><tt> <request name="destroy" type="destructor"></tt><tt> </tt><tt> <description summary="destroy presentation time request"></tt><tt> </tt><tt> The presentation_time object is destroyed and none of its events will</tt><tt> </tt><tt> ever be called again.</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> </request></tt><tt> </tt><tt> </tt><tt> <event name="presented"></tt><tt> </tt><tt> <description summary="buffer was displayed"></tt><tt> </tt><tt> Tell the client that a buffer was presented at time T=tv_sec+tv_nsec.</tt><tt> </tt><tt> This event only pertains to the one wl_buffer that was passed in when</tt><tt> </tt><tt> calling the buffer_queue.queue Thus, a particular presentation time</tt><tt> </tt><tt> feeback event is always associated with specific wl_buffer lifetime in</tt><tt> </tt><tt> the buffer queue.</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> <arg name="tv_sec" type="uint"</tt><tt> </tt><tt> summary="seconds value of the buffer presentation timestamp"/></tt><tt> </tt><tt> <arg name="tv_nsec" type="uint"</tt><tt> </tt><tt> summary="nanoseconds value of the buffer presentation timestamp"/></tt><tt> </tt><tt> </event></tt><tt> </tt><tt> </tt><tt> <event name="discarded"></tt><tt> </tt><tt> <description summary="buffer was not displayed"></tt><tt> </tt><tt> The buffer scheduled for presentation was not displayed and was</tt><tt> </tt><tt> removed from the buffer_queue.</tt><tt> </tt><tt> </description></tt><tt> </tt><tt> </event></tt><tt> </tt><tt> </interface></tt><tt> </tt><tt></protocol></tt> <div class="moz-signature"> <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"> <title></title> <a href="http://ca.linkedin.com/in/fredericplourde"><img alt="" src="cid:part1.07020704.08010709@collabora.co.uk" height="15" border="0" width="15"></a> Frédéric Plourde <div class="moz-signature"> Senior Software engineer <blockquote> T :: (450) 415-0855 @:: <a class="moz-txt-link-abbreviated" href="mailto:frederic.plourde@collabora.co.uk">frederic.plourde@collabora.co.uk</a> </blockquote> </div> </div> On 13-11-02 07:39 PM, Axel Davy wrote: </div> <blockquote cite="mid:1383435598-3833-1-git-send-email-axel.davy@ens.fr" type="cite"> <pre wrap="">Here is another proposition to extend the protocol to allow video players to control better the time at which frames hit the screen. Contrary to the original proposition: <a class="moz-txt-link-freetext" href="http://lists.freedesktop.org/archives/wayland-devel/2013-October/011496.html">http://lists.freedesktop.org/archives/wayland-devel/2013-October/011496.html</a> I do not separate the new request in another interface. The first request: presentation_time is only an indication to the compositor, and the hit request is a similar feedback to the frame request, but the notification is when the frame hit the screen. That's why I don't think it should be difficult for some compositors to implement it, and don't feel the need to separate it in another interface. The requests were designed to help implement the Present extension for XWayland. However the Present extension use Microseconds times, instead of Milliseconds. Any feedback is appreciated. I'm not English native, so if you feel some sentence are incorrect, please help me to correct them. Axel Davy (1): Add presentation_time and hit requests to wl_surface. protocol/wayland.xml | 70 +++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 67 insertions(+), 3 deletions(-) </pre> </blockquote> </body> </html>