Buffer Filters

[Pekka Paalanen]

On Fri, 26 Apr 2013 09:48:19 -0500
Jason Ekstrand <jason at jlekstrand.net> wrote:

> Wayland Devs,
> 
> What follows is an idea that's been kicking around my head for a little
> while now due to a discussion on the list a month or so ago.  The basic
> idea is to add an interface that provides some additional buffer types that
> don't have any back-end data of their own, but act as filters for other
> buffers.
> 
> As you would expect, the interface would have an enum representing all of
> the filters supported by the compositor and an event that would announce
> all of the supported filters upon binding by the client.  Then, there would
> be a series of "create_" requests that would take a new_id of type
> wl_buffer (for the newly created filter), a object argument of type
> wl_buffer for the buffer being wrapped, and possibly some other arguments
> depending on the filter.  In theory, you could stack filters, but that
> might get dicey and we would have to define some semantics for that.  I'm
> sure there are more filters that would be useful, but here are a couple
> that come to mind.
> 
> INVISIBLE
> 
> This is not really a filter.  Rather, it would be a buffer that has a size,
> but no content whatsoever.  The compositor would know that the surface to
> which this buffer is attached and, while it would be mapped onscreen would
> never be rendered.
> 
> I can think of a few different possible uses for this (mostly having to do
> with sub-ssurfaces).  For example, you could use an invisible buffer the
> size of your entire window to catch all of the input events and then patch
> the window together out of input-unaware subsurfaces.  I don't remember all
> of the details of subsurfaces and input, but this seems useful to me.

I don't really like the idea of a shortcut invisible buffer. The
problems you pointed out with sub-surfaces and the surface tree
structure; I have started to think all those difficulties are due
to not having a protocol object for a window. A window object could
be just a container for wl_surfaces, without any of its own
content, but adding that now would be too disruptive, I guess.

> SCALE/ZOOM
> 
> This would create a buffer whose content is provided by another buffer but
> has a different size.  The compositor would scale via GL or hardware
> overlays.  We would probably need an additional argument for the type of
> scaling (linear, cubic, etc.)
> 
> One possible use for this would be, for example, a video player.  A
> standard DVD video is encoded at 720x480 but is *always* scaled (720x480 is
> neither 16:9 or 4:3).  This would allow a media player to provide a 720x480
> video stream at 24 or 30 FPS and then have the compositor scale it up to
> 1920x1080 or whatever.  For fullscreen surfaces, this isn't a problem, but
> if they have a non-fullscreen but still big window, the client currently
> has to provide it at full resolution.
> 
> This has huge bandwidth advantages if the client is using wl_shm.  Also, it
> would allow the compositor to use hardware overlays when available to do
> the scaling which are (from what I know) more efficient and often provide
> better scaling than a client will get even with GL.

FWIW, on Friday afternoon I already wrote half of the XML protocol
spec for a crop & scale extension for wl_surfaces. The basic idea
is similar to sub-surfaces and shell surfaces: a global factory,
which you use to create an additional interface object for a
wl_surface. This interface object then allows you to set the source
rectangle to be read from a buffer, and destination size, which
will become the surface size on the next commit.

> This is just a quick list of a couple ideas.  Perhaps more immediate
> questions are, Is this useful? and, Is this too much of an abuse of
> wl_buffer?

For the use cases you mentioned, I am sceptical. It would certainly
be complex. For instance, as you have wl_buffers referencing other
wl_buffers, how do you implement buffer releasing logic? The
logical possibility of stacking you mentioned is also a problem: if
it is not supported, do we need a non-fatal way to notify about it?
If we do, that essentially becomes a roundtrip requirement: "did
this operation succeed?"

Right now, the wl_buffer reservance logic is fairly simple. A
committed attach makes a wl_buffer reserved by the server, and when
the server is guaranteed to not need the buffer anymore, it sends
the release event. If we have wl_buffers referencing wl_buffers,
when does a wl_buffer become reserved or released?

I would need to see more compelling use cases for your scheme to
be able to support it.


Thanks,
pq