[RFC weston 13/16] compositor: Add a function to test if images transformed by a matrix should be bilinearly filtered

[Pekka Paalanen]

On Wed, 1 Oct 2014 18:09:32 -0700
Jason Ekstrand <jason at jlekstrand.net> wrote:

> Allow me to chip in here.  Sorry that I haven't had a chance to really look
> over things carefully.  I have been reading this thread, just haven't had a
> chance to respond.
> 
> On Wed, Oct 1, 2014 at 12:41 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> 
> > On Tue, 30 Sep 2014 14:35:24 -0500
> > Derek Foreman <derekf at osg.samsung.com> wrote:
> >
> > > Thanks for taking a look!
> > >
> > > On 26/09/14 05:48 PM, Bill Spitzak wrote:
> > > > 90 degree rotation about x or y will require filtering.
> > >
> > > Yup, you're right.
> > >
> > > > You test y scale twice, must be a typo. I think you intended to test z,
> > > > but in fact z scale is not relevant so you should not test it at all.
> > >
> > > Argh - thanks.  Why isn't Z scale relevant?  I'm worried about making
> > > assumptions about the transformations these matrices represent and
> > > having those assumptions violated in the future...  For Z not to matter
> > > are we assuming projection will always be orthographic?
> >
> > Weston never uses the final Z coordinate for anything, so in that sense
> > it is always orthographic. Essentially, we could just do with 3x3
> > matrices perfectly fine. 3x3 supports 2D-projective which is enough to
> > implement fake-3D effects like
> > http://people.collabora.com/~pq/rotate3d-fun.webm
> > (The gl-renderer does not route the W element at all, I had to patch
> > that. Pixman-renderer OTOH just worked.)
> >
> > Weston also hardcodes the input Z coordinate always to 0, no matter
> > which way you are going between buffer and output spaces.
> >
> > I suppose the 4x4 matrix was originally chosen to fit the OpenGL API.
> > And maybe with some speculation about a desktop cube implementation or
> > something, but I don't really see the cube or such coming, not as a
> > generic thing anyway as only the gl-renderer could support it with
> > true 3D space.
> >
> > > > Translation by non-integer will also require filtering.
> > >
> > > Good point.
> > >
> > > > I recommend instead of checking the rotation to instead look for zeros
> > > > in the correct locations in the matrix. Matrix must be of the form:
> > > >
> > > >  |sx 0  0 tx|
> > > >  |0  sy 0 ty|
> > > >  |?  ?  ?  ?|
> > > >  |0  0  0  1|
> > > >
> > > > or
> > > >
> > > >  |0  sx 0 tx|
> > > >  |sy 0  0 ty|
> > > >  |?  ?  ?  ?|
> > > >  |0  0  0  1|
> > > >
> > > > sx and sy must be ±1, and tx and ty must be integers. The ? can be any
> > > > value.
> > >
> > > That could save us the very expensive matrix decomposition.  I'll try
> > > this out.  Thanks.
> > >
> > > I think this may be better than decomposition for deciding to use video
> > > planes in compositor-drm as well.
> > >
> > > (In fact, you've got me wondering if we ever need to split a matrix into
> > > basic transformations at all...)
> >
> > I'd be wondering about that, too. My intuition would say there is no
> > need to really decompose. Just checking the elements should suffice,
> > and when the matrix is supportable for whatever, then you pick the
> > right elements (which is a bit like decomposition, but no need to be
> > generic at all).
> >
> 
> Yeah, I'm not convinced we need to be able to do a full decomposition
> either.  My original intention was something like this:
> 
> bool
> weston_matrix_to_integer_transform(const weston_matrix *mat, enum
> wl_output_transform& transform, int *scale, int *x, int *y)

Why would there be 'transform' parameter? That implies that the matrix
is not really the total transformation, which I find odd here.

(Total transformation is between buffer pixel coords and output/scanout
pixel coords, i.e. buffer-to-output.)

> (do we use "bool" in weston?  Maybe just return int).  We may need both x
> and y scales and it may be useful to get those as floats.  I'm not sure on
> that.  Pekka, what would the RPi backend use?

The rpi-renderer uses pretty much the same as what DRM planes/overlays
offer wrt. coordinates, IIRC: integer position and size on the output,
and 16.16 fixed point position and size on input (buffer).

Whether scaling factor is integer or not is irrelevant there. I do not
recall there being an option for sampling (nearest/linear/...) in
either DispmanX nor DRM.

>  Basically, we want to be
> able to do 2 things: First, detect if it's an entirely integer transform
> and use GL_NEAREST instead of GL_LINEAR and second, know how to flip the
> surface around in cases when we can do some simple transformations but
> can't do an arbitrary matrix transformation.  One example here is DRM
> planes.  We can only use a plane when there's no scale and the
> buffer-to-output transform has no rotation.  We need to check for that
> condition and then pull the needed data out.

I think DRM planes do handle at least limited scaling, as do DispmanX
(IIRC something like if scaling to less than 1/8 you take an
additional performance hit, or other funny effects) also.

We may not know all the limitations of a DRM plane in advance, so we
only need to make sure it can fit through the KMS API, and then the
kernel will reject it if it violates some hw-specific restrictions.
(Fallback will be implemented in Weston when atomic/nuclear support
arrives.)

> Point is, we don't need a full matrix decomposition.  Also, it's worth
> throwing out there that the caching probably doesn't help us at all because
> we're going to usually be calling this on freshly computed matrices such as
> the above mentioned buffer-to-output transform.
> 
> Does that make sense?

Yes, to me at least.

Futhermore, if you wanted to cache the buffer-to-output matrix, you
would end up with number_of_views * number_of_outputs matrices to be
cached. The buffer-to-global per-view matrix might not change too
often, but we tend to paint outputs in turns, which means doing just
per-view cached total matrix is a waste.

So you might have buffer-to-surface matrix in weston_surface, then
buffer-to-global matrix cached in weston_view. I'm not sure it makes
sense to cache buffer-to-output anywhere.


Thanks,
pq