[RFC v.2] Extend wl_surface protocol
Pekka Paalanen
ppaalanen at gmail.com
Mon Nov 11 07:52:26 PST 2013
Hi Axel
On Fri, 08 Nov 2013 23:49:25 +0100
Axel Davy <davy at clipper.ens.fr> wrote:
> Hello,
>
> I've read carefully your new protocol proposition,
> but I think it doesn't meet the requirements to implement
> the X Present extension for XWayland.
>
> The problem is that I need to be able to use the frame
> request too (I need the frame callback and the presentation time).
Why do you need the frame callback? Is it to know when your one queued
buffer gets into use? That's the idea got from below.
And you specifically do not want to wait so long, that the presentation
timestamp arrives?
> Except this problem, I think your protocol proposition is fine.
> I suggest to change the spec
> to include the fact that queue is a more complete commit,
> and will take into account a pending frame request,
> and associate it to the wl_buffer we queue.
> Since the frame request is linked to a callback, the client
> can find to which buffer it is associated when it gets the
> frame feedback.
I don't think it makes sense to say, that "When the compositor
applies a buffer from the buffer queue, it will also implicitly commit
all pending frame requests." That's implicit behaviour across two
different interfaces in a rather surprising way, IMHO.
Stepping a bit into the unspecified(?) behaviour of the frame request,
you could submit your wl_buffer via wl_buffer_queue, and then issue the
normal wl_surface.frame with a wl_surface.commit. I believe the current
Weston implementation will trigger the frame callbacks even without a
new buffer, as long as something causes a repaint. Or the other way
around, I'm not sure which order is better for you.
> Note: to be able to get the presentation time, when the X client
> doesn't ask a specific presentation time, I'll have to use a queue
> of length one with a requested time of 0. Your spec says it should
> behave correctly.
Would you be using the queue length one only so that you know which
wl_buffer corresponds to the frame callback?
If you had frame callbacks tied to a particular submission of a
wl_buffer by some other means, would things be easier for you in
xwayland? Would be able to queue more buffers than just one? If yes,
would it benefit anything?
If it was really useful to know when each wl_buffer enters a
compositor repaint (just like what the frame request offers for
wl_surfaces), we could think about adding such an event to
wl_presentation_time interface (and maybe rename it accordingly).
>
> I'm adding a few comments behind.
>
> Axel Davy
>
> On 08/11/2013, Frederic Plourde wrote :
> > Hi,
> >
> > I have gathered comments and suggestions from colleagues and
> > wayland-devel reviewers and here is RFC v.2 of our buffer queue +
> > presentation feedback protocol extension.
> >
> > Notice the following changes :
> > -----------------------------------------
> > - it's changed name to make it more general. (comments/ideas on the
> > protocol and interfaces names are welcome). We believe this wl_surface
> > extension really is down to adding a timed buffer queue, hence the
> > wl_buffer_queue interface name.
> >
> > - it's been extended to include presentation time feedback
> > (wl_presentation_time)
> >
> > - Extensive comments were added.
> >
> > please, see the code below:
> >
> >
> >
> > <?xml version="1.0" encoding="UTF-8"?>
> > <protocol name="presentation_timing">
> >
> > <copyright>
> > Copyright © 2012-2013 Collabora, Ltd.
> >
> > Permission to use, copy, modify, distribute, and sell this
> > software and its documentation for any purpose is hereby granted
> > without fee, provided that the above copyright notice appear in
> > all copies and that both that copyright notice and this permission
> > notice appear in supporting documentation, and that the name of
> > the copyright holders not be used in advertising or publicity
> > pertaining to distribution of the software without specific,
> > written prior permission. The copyright holders make no
> > representations about the suitability of this software for any
> > purpose. It is provided "as is" without express or implied
> > warranty.
> >
> > THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS
> > SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
> > FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY
> > SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
> > WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
> > AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
> > ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
> > THIS SOFTWARE.
> > </copyright>
> >
> > <interface name="wl_presentation" version="1">
> > <description summary="buffer queues with presentation timing
> > information/
> > for surfaces">
> > The global factory interface exposing timestamped buffer queuing
> > composi-
> > ting capabilities.
> >
> > The aim of presentation timing is to support streaming video
> > generally
> > coming from videosink clients that typically need to queue video
> > buffers
> > with presentation timestamps in order to accurately synchronize
> > video
> > and audio streams.
> >
> > This global interface allows for the creation of timestamped buffer
> > queues for wl_surface objects.
> > </description>
> >
> > <request name="destroy" type="destructor">
> > <description summary="unbind from the wl_presentation interface">
> > Tell the server that the client will not be using this
> > protocol object anymore. This does not affect any other
> > objects, wl_buffer_queue objects included.
> > </description>
> > </request>
> >
> > <enum name="error">
> > <entry name="buffer_queue_exists" value="0"
> > summary="the surface already has a buffer_queue object/
> > associated to it"/>
> > </enum>
> >
> > <event name="clock_id", type="uint">
> > <description summary="clock ID to use">
> > Tell the client which clock ID is the compositor going to use
> > for time-
> > stamps and presentation feedback.
> >
> > Compositor sends that event once to a client right after it
> > binds to
> > the global interface.
> > </description>
> > </event>
> >
> Could you be more precise? To what corresponds the IDs? If I'm a client,
> and I have the
> choice between some IDs, which one should I take? Will the IDs mean the
> same on all Wayland
> compositors?
These are meant to be platform specific. For Linux, I would expect them
to be the valid clockid_t to clock_gettime(). The compositor will send
exactly one ID, and that clock shall be used in all timestamps related
to the buffer queue interfaces.
I'm not sure where we should be specifying platform-dependant
information, but I guess here would be a good place. So this still
needs to be added to the spec.
The alternative would be to enumerate all usable clocks over all
platforms, starting with the best on Linux and adding others as new
platforms come along. Not sure which is better.
> > <request name="create_queue">
> > <description summary="add buffer queueing capabilities to a
> > wl_surface">
> > Create and attach a wl_buffer_queue interface to the given wl_sur-
> > face. This effectively adds buffer queueing and presentation
> > timing
> > feedback capabilities to the surface through the use of buffer
> > queues
> > and presentation callbacks.
> >
> > If the given wl_surface already has a wl_buffer_queue object
> > associated
> > to it, the buffer_queue_exists protocol error is raised.
> >
> > Here, clock_id is needed to negociate time domain with the
> > compositor
> > as a common basis when specifing timestamps and presentation
> > callbacks.
> > clock_id is an implementation-specific integer representing
> > the clock
> > identification.
> > </description>
> >
> > <arg name="id" type="new_id" interface="wl_buffer_queue"
> > summary="the new buffer_queue object id"/>
> > <arg name="surface" type="object" interface="wl_surface"
> > summary="the surface to be turned intio a buffer_queue
> > surface"/>
> > <arg name="clock_id" type="uint" summary="clock ID to be used for/
> > timestamps">
You can ignore the clock id here, btw.
> > </request>
> > </interface>
> >
> > <interface name="wl_buffer_queue" version="1">
> > <description summary="buffer queue interface to a wl_surface">
> > An additional interface to a wl_surface object to enable buffer
> > queueing
> > capabilities. A buffer_queue-enabled surface contains a buffer
> > queue and
> > exposes protocol that allows for queuing timestamped wl_buffer
> > objects
> > into the queue and requesting presentation time feedback.
> >
> > Buffer_queue-enabled surfaces do not declare any state in
> > addition to the
> > ones wl_surfaces already use by default. Thus, once a plain
> > wl_surface has
> > been turned into a buffer_queue-enabled surface, buffer queuing hap-
> > pens through the "queue" request. This means that queuing up
> > buffers does *not* require any surface.attach nor
> > surface.commit. More-
> > over, such an attach + commit sequence will clear the buffer
> > queue in
> > order to exclusively consider the newly attached buffer. Hence,
> > surface.attach and surface.commit should *not* be used when
> > specifying
> > streaming surface content via buffer_queue. Also, queuing up a
> > wl_buffer
> > makes it "reserved" in the compositor just like attach + commit
> > does.
> >
> > Besides, it is perfectly acceptable to commit (without attach) some
> > states on a buffer_queue-enabled surface, as it will not clear
> > the buffer
> > queue nor interfere with any presentation timing mechanism
> > brought by
> > the presentation_timing interfaces.
> >
> > The compositor will attempt to repaint so that each queued
> > buffer gets
> > presented at the requested target time. However, this may not
> > always be
> > possible e.g. when a requested presentation time cannot be met
> > accurately
> > or when the target time has completely passed already and been
> > replaced by
> > another buffer by the time the compositor can repaint again.
> > Therefore,
> > At every repaint cycle, enqueued buffers with past-timestamps,
> > if any,
> > will be considered and the compositor will present the most
> > recent one
> > among them. At that point, buffers with older timestamps will be
> > removed
> > from the queue and released (a BUFFER_RELEASE event will be
> > fired for
> > every removed one). Also, at every repaint cycle, future-timestamped
> > buffers will be kept untouched in the queue.
> >
> > When applying a buffer from the queue, the compositor implies full
> > surface damage. If the client manages to attach + damage +
> > commit in the
> > meantime, then the damage is what it explicitely specified (and the
> > buffer_queue gets cleared)
> > </description>
> >
> > <request name="destroy" type="destructor">
> > <description summary="remove buffer_queue interface">
> > The buffer_queue interface is removed from the
> > buffer_queue-enabled
> > surface.
> > </description>
> > </request>
> >
> > <request name="queue">
> > <description summary="enqueue a buffer into the surface's buffer
> > queue">
> > Queue a buffer along with its timestamp into the surface's
> > internal
> > buffer queue. This timestamp is the intended presentation
> > time. While
> > clients generally want to queue posterior timestamps (future),
> > queueing
> > up anterior timestamps (in the past) is allowed and does not
> > raise any
> > protocol error.
> >
> > The speficied timestamp is two-part. tv_sec is the number of
> > seconds
> > and tv_nsec defines the number of nanoseconds spent after tv_sec.
> >
> > Calling queue() will create a wl_presentation_time object that
> > is ex-
> > clusively associated with the provided wl_buffer lifetime in
> > the queue
> > at the moment the object was created. This means that presentation
> > feedback concerning this wl_buffer will be provided through
> > wl_presen-
> > tation_time events, but only once. As soon as the buffer is
> > presented
> > and removed from the queue by the compositor, it's internal
> > link to
> > the wl_presentation_time object is broken and associated
> > presentation
> > events will never be called again. So a client, right after
> > having recei-
> > ved a wl_presentation_time event will generally destroy that
> > object.
> > On the other hand, if a client does not want feedback, it may
> > destroy
> > presentation_time object right after queuing a buffer.
> > </description>
> >
> > <arg name="buffer" type="object" interface="wl_buffer"
> > summary="the buffer to queue"/>
> > <arg name="id" type="new_id" interface="wl_presentation_time"
> > summary="the new presentation_time object id"/>
> > <arg name="tv_sec" type="uint"
> > summary="seconds value of the buffer target timestamp"/>
> > <arg name="tv_nsec" type="uint"
> > summary="nanoseconds value of the buffer target timestamp"/>
> > </request>
> >
> > <request name="clear">
> > <description summary="clear the surface's buffer queue">
> > Clear the surface's buffer queue. All buffers are removed from the
> > queue and released as appropriate. The requested presentation
> > times are
> > discarded. The surface retains the contents it currently has,
> > at the
> > time the compositor processes this request. The compositor may not
> > release the buffer(s) currently being displayed if it is
> > needed for
> > repainting or scanout, as usual.
> > </description>
> > </request>
> > </interface>
> >
> > <interface name="wl_presentation_time" version="1">
> > <description summary="presentation time feedback event">
> > The wl_presentation_time object encapsulates presentation time
> > events
> > sent as feedback by the compositor to a client that previously
> > queued
> > a wl_buffer. wl_presentation_time objects are created and
> > returned when
> > a client enqueues a buffer through the wl_buffer_queue.queue()
> > request.
> >
> > Note that presentation time only tells client when the
> > compositor presen-
> > ted the buffer to display hardware, not when the buffer was
> > turned into
> > light (actually displayed on screen) by this hardware. As there
> > could
> > be anything displaying those buffers, from very fast, low-latency
> > computer monitors to slow, hi-latency HDMI TV screens, it is the
> > client's
> > responsibility to make sure it knows what display hardware is
> > currently
> > connected and what is its latency.
> > </description>
> It should be even more precise here, else we might not see the
> difference with
> the frame request (which, by the way, should be clarified).
>
> For example when you schedule a pageflip, you would send the presented
> event only when you know
> the pageflip has succeeded, not when you schedule the pageflip.
> I believe "when the compositor presented the buffer to display hardware"
> could be interpreted the wrong way in this example (scheduling a
> pageflip can
> be considered presenting the buffer to display hardware).
Yeah, that's just unclear wording.
> >
> > <request name="destroy" type="destructor">
> > <description summary="destroy presentation time request">
> > The presentation_time object is destroyed and none of its
> > events will
> > ever be called again.
> > </description>
> > </request>
> >
> > <event name="presented">
> > <description summary="buffer was displayed">
> > Tell the client that a buffer was presented at time
> > T=tv_sec+tv_nsec.
> > This event only pertains to the one wl_buffer that was passed
> > in when
> > calling the buffer_queue.queue Thus, a particular
> > presentation time
> > feeback event is always associated with specific wl_buffer
> > lifetime in
> > the buffer queue.
> > </description>
> > <arg name="tv_sec" type="uint"
> > summary="seconds value of the buffer presentation timestamp"/>
> > <arg name="tv_nsec" type="uint"
> > summary="nanoseconds value of the buffer presentation
> > timestamp"/>
> > </event>
> It should be precised that tv_nsec won't go over 999,999,999.
How about we say "at time given as tv_sec seconds and the fractional
part in tv_nsec nanoseconds." Though, even the man-page does not seem
to explicitly say that.
> > <event name="discarded">
> > <description summary="buffer was not displayed">
> > The buffer scheduled for presentation was not displayed and was
> > removed from the buffer_queue.
> > </description>
> > </event>
> > </interface>
> > </protocol>
> >
Thanks,
pq
More information about the wayland-devel
mailing list