Buffer Filters

Sun Apr 28 07:41:03 PDT 2013

On Sun, Apr 28, 2013 at 1:24 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:

> On Fri, 26 Apr 2013 09:48:19 -0500
> Jason Ekstrand <jason at jlekstrand.net> wrote:
>
> > Wayland Devs,
> >
> > What follows is an idea that's been kicking around my head for a little
> > while now due to a discussion on the list a month or so ago.  The basic
> > idea is to add an interface that provides some additional buffer types
> that
> > don't have any back-end data of their own, but act as filters for other
> > buffers.
> >
> > As you would expect, the interface would have an enum representing all of
> > the filters supported by the compositor and an event that would announce
> > all of the supported filters upon binding by the client.  Then, there
> would
> > be a series of "create_" requests that would take a new_id of type
> > wl_buffer (for the newly created filter), a object argument of type
> > wl_buffer for the buffer being wrapped, and possibly some other arguments
> > depending on the filter.  In theory, you could stack filters, but that
> > might get dicey and we would have to define some semantics for that.  I'm
> > sure there are more filters that would be useful, but here are a couple
> > that come to mind.
> >
> > INVISIBLE
> >
> > This is not really a filter.  Rather, it would be a buffer that has a
> size,
> > but no content whatsoever.  The compositor would know that the surface to
> > which this buffer is attached and, while it would be mapped onscreen
> would
> > never be rendered.
> >
> > I can think of a few different possible uses for this (mostly having to
> do
> > with sub-ssurfaces).  For example, you could use an invisible buffer the
> > size of your entire window to catch all of the input events and then
> patch
> > the window together out of input-unaware subsurfaces.  I don't remember
> all
> > of the details of subsurfaces and input, but this seems useful to me.
>
> I don't really like the idea of a shortcut invisible buffer. The
> problems you pointed out with sub-surfaces and the surface tree
> structure; I have started to think all those difficulties are due
> to not having a protocol object for a window. A window object could
> be just a container for wl_surfaces, without any of its own
> content, but adding that now would be too disruptive, I guess.
>

I would like to see some sort of invisible surface for use with
sub-surfaces as it makes the surface trees much nicer.  That said, there
may be a better way to handle it than this.  Perhaps simply an extension of
wl_surface that allows you to map with a particular size and no buffer.

> > SCALE/ZOOM
> >
> > This would create a buffer whose content is provided by another buffer
> but
> > has a different size.  The compositor would scale via GL or hardware
> > overlays.  We would probably need an additional argument for the type of
> > scaling (linear, cubic, etc.)
> >
> > One possible use for this would be, for example, a video player.  A
> > standard DVD video is encoded at 720x480 but is *always* scaled (720x480
> is
> > neither 16:9 or 4:3).  This would allow a media player to provide a
> 720x480
> > video stream at 24 or 30 FPS and then have the compositor scale it up to
> > 1920x1080 or whatever.  For fullscreen surfaces, this isn't a problem,
> but
> > if they have a non-fullscreen but still big window, the client currently
> > has to provide it at full resolution.
> >
> > This has huge bandwidth advantages if the client is using wl_shm.  Also,
> it
> > would allow the compositor to use hardware overlays when available to do
> > the scaling which are (from what I know) more efficient and often provide
> > better scaling than a client will get even with GL.
>
> FWIW, on Friday afternoon I already wrote half of the XML protocol
> spec for a crop & scale extension for wl_surfaces. The basic idea
> is similar to sub-surfaces and shell surfaces: a global factory,
> which you use to create an additional interface object for a
> wl_surface. This interface object then allows you to set the source
> rectangle to be read from a buffer, and destination size, which
> will become the surface size on the next commit.
>

Perhaps this is something better handled at the surface level.  The primary
reason why I was suggesting doing it with buffers is because, in the
protocol, surfaces get their sizes from buffers.  That said, I'm eager to
see (even just the XML) for your wl_surface method of handling it.  If we
can make that work well, it very well may be cleaner particularly because
you wouldn't need a stack of filter buffers for each buffer, you just need
the one crop/scale for the surface.

> > This is just a quick list of a couple ideas.  Perhaps more immediate
> > questions are, Is this useful? and, Is this too much of an abuse of
> > wl_buffer?
>
> For the use cases you mentioned, I am sceptical. It would certainly
> be complex. For instance, as you have wl_buffers referencing other
> wl_buffers, how do you implement buffer releasing logic? The
> logical possibility of stacking you mentioned is also a problem: if
> it is not supported, do we need a non-fatal way to notify about it?
> If we do, that essentially becomes a roundtrip requirement: "did
> this operation succeed?"
>
> Right now, the wl_buffer reservance logic is fairly simple. A
> committed attach makes a wl_buffer reserved by the server, and when
> the server is guaranteed to not need the buffer anymore, it sends
> the release event. If we have wl_buffers referencing wl_buffers,
> when does a wl_buffer become reserved or released?
>

Server-side, this would be as simple as reference counting.  On the client
side, you're right that it gets more complicated.  In most use-cases where
you have one buffer per surface, it would be exactly the same; you would
simply recieve multiple events.

> I would need to see more compelling use cases for your scheme to
> be able to support it.
>

Right.  Stacking buffers provides a nice, theoretically simple, way to
handle these types of things.  However, it does make for a lot of
edge-cases in implementation.

Thanks,
--Jason Ekstrand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20130428/a43463b9/attachment-0001.html>