[PATCH 2/2] protocol: Support scaled outputs and surfaces

Wed May 22 23:29:30 PDT 2013

On Wed, 22 May 2013 21:49:25 -0400
Kristian Høgsberg <hoegsberg at gmail.com> wrote:

> On Mon, May 20, 2013 at 10:49:27AM +0300, Pekka Paalanen wrote:
> > Hi Alexander,
> > 
> > nice to see this going forward, and sorry for replying so rarely and
> > late.
> > 
> > On Thu, 16 May 2013 15:49:36 +0200
> > alexl at redhat.com wrote:
> > 
> > > From: Alexander Larsson <alexl at redhat.com>
> > > 
> > > This adds the wl_surface.set_buffer_scale request, and a wl_output.scale
> > > event. These together lets us support automatic upscaling of "old"
> > > clients on very high resolution monitors, while allowing "new" clients
> > > to take advantage of this to render at the higher resolution when the
> > > surface is displayed on the scaled output.
> > > 
> > > It is similar to set_buffer_transform in that the buffer is stored in
> > > a transformed pixels (in this case scaled). This means that if an output
> > > is scaled we can directly use the pre-scaled buffer with additional data,
> > > rather than having to scale it.
> > > 
> > > Additionally this adds a "scaled" flag to the wl_output.mode flags
> > > so that clients know which resolutions are native and which are scaled.
> > > 
> > > Also, in places where the documentation was previously not clear as to
> > > what coordinate system was used this was fleshed out.
> > > 
> > > It also adds a scaling_factor event to wl_output that specifies the
> > > scaling of an output.
> > > 
> > > This is meant to be used for outputs with a very high DPI to tell the
> > > client that this particular output has subpixel precision. Coordinates
> > > in other parts of the protocol, like input events, relative window
> > > positioning and output positioning are still in the compositor space
> > 
> > We don't have a single "compositor space".
> > 
> > This needs some way of explaining that surface coordinates are always
> > the same, regardless of the attached buffer's scale. That is, surface
> > coordinates always correspond to the size of a buffer with scale 1.
> > 
> > > rather than the scaled space. However, input has subpixel precision
> > > so you can still get input at full resolution.
> > > 
> > > This setup means global properties like mouse acceleration/speed,
> > > pointer size, monitor geometry, etc can be specified in a "mostly
> > > similar" resolution even on a multimonitor setup where some monitors
> > > are low dpi and some are e.g. retina-class outputs.
> > > ---
> > >  protocol/wayland.xml | 107 ++++++++++++++++++++++++++++++++++++++++++++-------
> > >  1 file changed, 93 insertions(+), 14 deletions(-)
> > > 
> > > diff --git a/protocol/wayland.xml b/protocol/wayland.xml
> > > index d3ae149..acfb140 100644
> > > --- a/protocol/wayland.xml
> > > +++ b/protocol/wayland.xml
...
> > > @@ -860,6 +864,9 @@
> > >  
> > >  	The client is free to dismiss all but the last configure
> > >  	event it received.
> > > +
> > > +	The width and height arguments specify the size of the window
> > > +	in surface local coordinates.
> > 
> > Yes, "window" is definitely the correct term here. Saying "surface"
> > would be incorrect, due to sub-surfaces, if I recall my discussion with
> > Giulio Camuffo right.
> 
> I think we need to introduce the "window geometry" concept before this
> gets out of hand.  I'm already a little uncomfortable with the idea of
> including sub-surfaces in the surface bounding box - it seems like
> something the client should control.  So the suggestion is to add a
> new wl_shell_surface request:
> 
>     <request name="set_geometry">
>       <description summary="set logical surface geometry">
> 	This request sets the the logical geometry of the surface.
> 
> 	The logical geometry is the rectangle that compositor uses for
> 	window management placement decisions.  For example, a buffer
> 	often contains drop shadow for a surface or other content that
> 	shouldn't be considered part of the surface geometry.  By
> 	setting the surface geometry, the client can communicate to
> 	the compositor the sub-rectangle of the surface that it
> 	considers the window geometry.  The compositor will the use
> 	this rectangle for initial placement, snapping and other
> 	placement decisions.  If the client has provided a geometry
> 	for a surface, the new size suggested by the configure event
> 	will refer to the window geometry.
> 
> 	The geometry is a rectangle in surface local coordinates.
> 
> 	Geometry is double-buffered state, see wl_surface.commit.
> 
> 	wl_shell_surface.set_geometry changes the pending geometry.
> 	wl_surface.commit copies the pending geometry to the current
> 	geometry.  Otherwise, the pending and current geometry are
> 	never changed.
> 
> 	Initially the geometry will track the bounding box of the
>         surface but once the geometry is set, it will only change when
>         the geomtry is set again.
>       </description>
> 
>       <arg name="x" type="int"/>
>       <arg name="y" type="int"/>
>       <arg name="width" type="int"/>
>       <arg name="height" type="int"/>
>     </request>
> 

Yeah, that should work. Avoids nicely using the input region bounding
box as the surface extents for window management purposes.

> > >        </description>
> > >  
> > >        <arg name="edges" type="uint"/>
> > > @@ -876,11 +883,16 @@
> > >      </event>
> > >    </interface>
> > >  
> > > -  <interface name="wl_surface" version="2">
> > > +  <interface name="wl_surface" version="3">
> > >      <description summary="an onscreen surface">
> > >        A surface is a rectangular area that is displayed on the screen.
> > >        It has a location, size and pixel contents.
> > >  
> > > +      The size of a surface (and relative positions on it) is described
> > 
> > "The size of the surface and positions on the surface are described..."?
> > 
> > > +      in surface local coordinates, which may differ from the buffer
> > > +      local coordinates of the pixel content, in case a buffer_transform
> > > +      or a buffer_scale is used.
> > 
> > I think we could additionally define here, that surface local
> > coordinates always correspond to the case buffer_scale=1. Would that
> > define it once and for all?
> > 
> > > +
> > >        Surfaces are also used for some special purposes, e.g. as
> > >        cursor images for pointers, drag icons, etc.
> > >      </description>
> > > @@ -895,20 +907,25 @@
> > >        <description summary="set the surface contents">
> > >  	Set a buffer as the content of this surface.
> > >  
> > > +	The new size of the surface is calculated based on the buffer
> > > +	size transformed by the inverse buffer_transform and the
> > > +	inverse buffer_scale. This means that the supplied buffer
> > > +	must be an integer multiple of the buffer_scale.
> > > +
> > >  	The x and y arguments specify the location of the new pending
> > > -	buffer's upper left corner, relative to the current buffer's
> > > -	upper left corner. In other words, the x and y, and the width
> > > -	and height of the wl_buffer together define in which directions
> > > -	the surface's size changes.
> > > +	buffer's upper left corner, relative to the current buffer's upper
> > > +	left corner, in surface local coordinates. In other words, the
> > > +	x and y, combined with the new surface size define in which
> > > +	directions the surface's size changes.
> > 
> > In this paragraph, the current definition (program state) of surface
> > coordinates is being changed: the origin is moved when this action is
> > applied. Therefore I originally did not mention surface coordinates
> > here, since we need to specify in which surface coordinates x,y are in,
> > old or new.
> > 
> > >  	Surface contents are double-buffered state, see wl_surface.commit.
> > >  
> > >  	The initial surface contents are void; there is no content.
> > >  	wl_surface.attach assigns the given wl_buffer as the pending
> > >  	wl_buffer. wl_surface.commit makes the pending wl_buffer the new
> > > -	surface contents, and the size of the surface becomes the size of
> > > -	the wl_buffer, as described above. After commit, there is no
> > > -	pending buffer until the next attach.
> > > +	surface contents, and the size of the surface becomes the size
> > > +	calculated from the wl_buffer, as described above. After commit,
> > > +	there is no pending buffer until the next attach.
> > 
> > Right.
> > 
> > >  	Committing a pending wl_buffer allows the compositor to read the
> > >  	pixels in the wl_buffer. The compositor may access the pixels at
> > > @@ -945,6 +962,8 @@
> > >  
> > >  	Damage is double-buffered state, see wl_surface.commit.
> > >  
> > > +	The damage rectangle is specified in surface local coordinates.
> > > +
> > >  	The initial value for pending damage is empty: no damage.
> > >  	wl_surface.damage adds pending damage: the new pending damage
> > >  	is the union of old pending damage and the given rectangle.
> > > @@ -996,6 +1015,8 @@
> > >  	behaviour, but marking transparent content as opaque will result
> > >  	in repaint artifacts.
> > >  
> > > +	The opaque region is specified in surface local coordinates.
> > > +
> > >  	The compositor ignores the parts of the opaque region that fall
> > >  	outside of the surface.
> > >  
> > > @@ -1023,6 +1044,8 @@
> > >  	surface in the server surface stack. The compositor ignores the
> > >  	parts of the input region that fall outside of the surface.
> > >  
> > > +	The input region is specified in surface local coordinates.
> > > +
> > >  	Input region is double-buffered state, see wl_surface.commit.
> > 
> > Alright.
> > 
> > >  	wl_surface.set_input_region changes the pending input region.
> > > @@ -1100,7 +1123,7 @@
> > >  	according to the output transform, thus permiting the compositor to
> > >  	use certain optimizations even if the display is rotated. Using
> > >  	hardware overlays and scanning out a client buffer for fullscreen
> > > -	surfaces are examples of such optmizations. Those optimizations are
> > > +	surfaces are examples of such optimizations. Those optimizations are
> > >  	highly dependent on the compositor implementation, so the use of this
> > >  	request should be considered on a case-by-case basis.
> > 
> > Heh, nice catch.
> > 
> > 
> > I think we should change the wl_surface.commit description, so that
> > buffer_transform and buffer_scale are taken into account first, then
> > apply the new wl_buffer, and then everything else. This is because the
> > new wl_buffer defines the surface coordinate system, where everything
> > else is then described in.
> 
> I don't think so.  The buffer doesn't change or define the surface
> coordinate system.  If you attach the same buffer with a different
> scale, or bigger or smaller buffer with the same scale, that just
> changes the surface size.  If you set an opaque region it doesn't
> matter whether you do it before or after the new buffer or scale
> factor is set, the region is what it is.

What about the attach arguments x and y?

Right, they are in surface coordinates, so buffer_scale and
buffer_transform do not affect their interpretation. They still move
the surface coordinate system, so at least it definitely matters
whether the given regions are set before or after. We also clip the
given regions to the surface size, so the surface size must be set
first, and that depends on the buffer_scale.

Actually... since x,y are defined to mean rows/columns to add or
remove, should we not translate existing regions according to them
when new regions are not given? By a vague recollection, we do not do
that in Weston, and toytoolkit just resets all regions whenever the size
may change.

Besides, we need something to define the surface coordinate system to
being with, and the only way to do that is to refer to the buffer, also
on the very first commit, where it will matter, whether we apply the
buffer_scale to the surface state before or after defining the surface
coordinate system by applying the buffer.

Or can we somehow hang the surface coordinate system in the air with
"pels"? That is, start the chain of definitions from surface coordinates
instead of the buffer coordinates?

Thanks,
pq