[RFC] wl_surface scale and crop protocol extension

[Pekka Paalanen]

On Tue, 30 Apr 2013 10:54:25 -0500
Jason Ekstrand <jason at jlekstrand.net> wrote:

> On Tue, Apr 30, 2013 at 9:06 AM, John Kåre Alsaker <
> john.kare.alsaker at gmail.com> wrote:
> 
> > I'd say we should split the cropping and scaling request into two.
> >
> 
> How would you suggest doing that?  I actually really like the
> simplicity of this box maps to that box.

The boxes approach also does not have rounding issues. The
destination size will be exactly what you specify, not some vague
number multiplied by number, then rounded somehow.

> > Specifying a scaling with a NULL buffer should still set the
> > surface size, so we can have surfaces with only an input region.
> >
> 
> I really don't think this is the place for that.  I want input-only
> surfaces too, but I don't think this is the place for that

I also don't think this is the place for that. If any...

> > I don't see a way to disable scaling and cropping, could passing 0
> > as width and height do that?
> >
> 
> Can't the client just set it to 0, 0, width, height, width, height
> easily enough?  Or just destroy the scaler?

Yeah, that's two ways to disable scaling and cropping, should be
enough. The first one just has to be used repeatedly, if the buffer
size changes. Destroying the surface_scaler returns the original 1:1
mapping.

> On Tue, Apr 30, 2013 at 2:33 PM, Pekka Paalanen <ppaalanen at gmail.com>
> wrote:
> 
> > Hi all,
> >
> > below is my first draft for a wl_surface scaling and cropping
> > extension. I called it wl_scaler in the lack of a better name. It is
> > designed similarly to the other wl_surface extensions
> > wl_shell_surface and wl_subsurface.
> >
> > There probably isn't any interesting details to debate, right? ;-)
> >
> > I'd like to have a better name for it, and you might want the set
> > request split into two or three, or not, but otherwise I'm quite
> > satisfied with it. I tried to take into account all existing
> > protocol that interacts with this.
> >
> > Comments? Is the language clear?
> >
> >
> > Thanks,
> > pq
> >
> >
> > <?xml version="1.0" encoding="UTF-8"?>
> > <protocol name="scaler">
> >
> >   <copyright>
> >     Copyright © 2013 Collabora, Ltd.
> >
> >     Permission to use, copy, modify, distribute, and sell this
> >     software and its documentation for any purpose is hereby granted
> >     without fee, provided that the above copyright notice appear in
> >     all copies and that both that copyright notice and this
> > permission notice appear in supporting documentation, and that the
> > name of the copyright holders not be used in advertising or
> > publicity pertaining to distribution of the software without
> > specific, written prior permission.  The copyright holders make no
> >     representations about the suitability of this software for any
> >     purpose.  It is provided "as is" without express or implied
> >     warranty.
> >
> >     THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO
> > THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
> > AND FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR
> > ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
> >     WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER
> > IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
> >     ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
> >     THIS SOFTWARE.
> >   </copyright>
> >
> >   <interface name="wl_scaler" version="1">
> >     <description summary="surface cropping and scaling">
> >       The global interface exposing surface cropping and scaling
> >       capabilities is used to instantiate an interface extension
> > for a wl_surface object. This extended interface will then allow
> >       cropping and scaling the surface contents, effectively
> >       disconnecting the 1:1 relationship between the buffer and the
> >       surface size.
> >     </description>
> >
> >     <request name="destroy" type="destructor">
> >       <description summary="unbind from the cropping and scaling
> > interface">
> >         Informs the server that the client will not be using this
> >         protocol object anymore. This does not affect any other
> > objects, wl_surface_scaler objects included.
> >       </description>
> >     </request>
> >
> >     <enum name="error">
> >       <entry name="scaler_exists" value="0"
> >              summary="the surface already has a scaler object
> > associated"/> </enum>
> >
> >     <request name="get_surface_scaler">
> >       <description summary="extend surface interface for crop and
> > scale"> Instantiate an interface extension for the given wl_surface
> > to crop and scale its content. If the given wl_surface already has
> >         a wl_surface_scaler object associated, the scaler_exists
> >         protocol error is raised.
> >       </description>
> >
> >       <arg name="id" type="new_id" interface="wl_surface_scaler"
> >            summary="the new scaler interface id"/>
> >       <arg name="surface" type="object" interface="wl_surface"
> >            summary="the surface"/>
> >     </request>
> >   </interface>
> >
> >   <interface name="wl_surface_scaler" version="1">
> >     <description summary="crop and scale interface to a wl_surface">
> >       An additional interface to a wl_surface object, allowing
> > cropping and scaling of the surface contents.
> >
> 
> "allows you" or "allows the client" would be better here

I changed the paragraph to:

"     An additional interface to a wl_surface object, which allows the
      client to specify the cropping and scaling of the surface
      contents."


> >       This interface allows to define the source rectangle (src_x,
> >       src_y, src_width, src_height) from where to take the wl_buffer
> >       contents, and scale that to destination size (dst_width,
> >       dst_height). This state is double-buffered, and is applied on
> > the next wl_surface.commit.
> >
> >       Before the first set request, the wl_surface still behaves as
> > if there was no crop and scale state. That is, no scaling is
> > applied, and the surface size is the buffer size.
> >
> >       On compositing, source rectangle coordinates are evaluated
> > after wl_surface.set_buffer_transform is evaluated. This means that
> >       changing the buffer transform and correspondingly the client
> >       rendering does not require sending new source rectangle
> >       coordinates to keep the exact same image source rectangle. In
> >       other words, the source rectangle is given in the
> >       not-scaled-and-cropped surface coordinates, not buffer data
> >       coordinates.
> >
> 
> I agree with Zhi, this needs to be re-worded

Yeah, did you understand what I was trying to explain? Any suggestions?

> >       The crop and scale state causes the surface size to become
> >       dst_width, dst_height. This overrides whatever the attached
> >       wl_buffer size is, unless the wl_buffer is NULL.
> >
> >       The surface-local coordinate system is tied to the space of
> >       dst_width and dst_height. That is, surface-local coordinates
> > are relative to the scaled and cropped surface, not the wl_buffer
> >       providing the content.
> >
> >       If the source rectangle is partially or completely outside of
> > the wl_buffer, then the surface contents are undefined, and the
> >       surface size is still dst_width, dst_height.
> >
> >       The x, y arguments of wl_surface.attach are applied as normal
> > to the surface. They indicate how many pixels to remove from the
> >       surface size from the left and the top. In other words, they
> > are still in the surface-local coordinate system, just like
> > dst_width and dst_height are.
> >

This above paragraph...

> >       If the wl_surface associated with the wl_surface_scaler is
> >       destroyed, the wl_surface_scaler object becomes inert.
> >
> >       If the wl_surface_scaler object is destroyed, the crop and
> > scale state is removed from the wl_surface. The change will be
> > applied on the next wl_surface.commit.
> >     </description>
> >
> >     <request name="destroy" type="destructor">
> >       <description summary="remove scaling and cropping from the
> > surface"> The associated wl_surface's crop and scale state is
> > removed. The change is applied on the next wl_surface.commit.
> >       </description>
> >     </request>
> >
> >     <enum name="error">
> >       <entry name="bad_value" value="0"
> >              summary="negative values in width or height"/>
> >     </enum>
> >
> >     <request name="set">
> >       <description summary="set the crop and scale state">
> >         Set the crop and scale state of the associated wl_surface.
> > See wl_surface_scaler for the description, and relation to the
> >         wl_buffer size.
> >
> >         If any of src_width, src_height, dst_width, and dst_height
> > is negative, the bad_value protocol error is raised.
> >       </description>
> >
> >       <arg name="src_x" type="fixed" summary="source rectangle x"/>
> >       <arg name="src_y" type="fixed" summary="source rectangle y"/>
> >       <arg name="src_width" type="fixed" summary="source rectangle
> > width"/> <arg name="src_height" type="fixed" summary="source
> > rectangle height"/>
> >       <arg name="dst_width" type="int" summary="surface width"/>
> >       <arg name="dst_height" type="int" summary="surface height"/>
> >     </request>
> >
> >   </interface>
> > </protocol>
> >
> 
> There's only one gaping hole that I see here and that is attach
> behaviour. Currently, wl_surface.attach takes an (x, y) position
> argument telling the compositor where to place the new buffer
> relative to the old one.  How is this going to interact with this
> extension?

...should explain the interaction precicely. Is it not understandable?

I explained it via adding/removing pixel rows or columns, as that
is the fundamental intention. Moving the surface is just a consequence.

> I think the problem here is the same one that we've seen before:
> Surfaces inherently take their size etc. from the buffer attached to
> them.  While this works great in the "single image exactly the size
> it should be on screen" case, I'm not sure it's working well when we
> go beyond that. Unfortunately, I'm not 100% sure how to solve it.

I hope I managed to solve it. :-)

However, my proposed solution relies on the wl_surface_scaler interface
being controlled by the same entity who renders and commits the
wl_buffers. If it is not supposed to be the same entity (library,
component), then we have more problems than just defining what
attach(x,y) does.

What I mean is, that this extension is not designed for an independent
component to force the size of another component's (sub-)surface
behind its back. wl_surface_scaler interface and wl_surface.attach
request must be used together for coherent and fully controlled
results. This idea also makes sampling from outside of a buffer just a
silent, non-fatal client error, rather than trying to come up with a
scheme to transparently fix such mismatches.


Thanks,
pq