surface buffer cardinality and outputs

[Jerome Glisse]

On Fri, Mar 22, 2013 at 5:51 PM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> On Fri, 22 Mar 2013 10:41:00 -0400
> Jerome Glisse <j.glisse at gmail.com> wrote:
>
>> On Fri, Mar 22, 2013 at 6:35 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
>> > On Wed, 20 Mar 2013 13:09:48 -0400
>> > Jerome Glisse <j.glisse at gmail.com> wrote:
>> >
>> >> On Wed, Mar 20, 2013 at 4:32 AM, RenoX <renozyx at gmail.com> wrote:
>> >> >
>> >> >
>> >> > On Tue, Mar 19, 2013 at 11:34 AM, Jason Ekstrand <jason at jlekstrand.net>
>> >> > wrote:
>> >> >> I'm not sure exactly what I think of all this surface transform
>> >> >> passing.  I'll get back to that once I get a chance to think about it.
>> >> >>  Part of the problem is that you don't want to try and make your
>> >> >> animations subpixel-perfect because that would require a lot of
>> >> >> round-trips to make it right.
>> >> >
>> >> > And you(Jerome Glisse) replied:
>> >> >> It doesn't add more roundtrip, client will get next matrix like a
>> >> >> frame event, client then draw in screen space and then draw with new
>> >> >> transformation matrix. If client can't keep up it wouldn't either keep
>> >> >> up in the non screen space rendering case.
>> >> >
>> >> > I disagree: currently Weston move windows server side only, that is to say
>> >> > without roundtrip,
>> >> > if you want to be 'subpixel perfect' then moving a window imply to re-render
>> >> > it from the client, which will add a lot of roundtrip in this case.
>> >> >
>> >> > So the current design is 'pixel perfect', but not 'subpixel perfect', and
>> >> > cannot be without big changes..
>> >> >
>> >> > regards,
>> >> > renoX
>> >>
>> >> What i propose is an extension ie application can still render non
>> >> subpixel and can still render non subpixel when moving/resizing. Note
>> >> also that moving a window around is not so much different then
>> >> resizing a window in which case wayland will round trip with the
>> >> client. So really if a client want subpixel rendering at all time it
>> >> just add roundtrip in the window is moving case.
>> >
>> > The server can also move surfaces on its own, without the client
>> > initiating the move. This is move by hotkey. Therefore you would
>> > incur at least one extra roundtrip during the beginning of such move.
>> >
>> >> Again i stress it's not a lot of roundtrip it's the same amount of
>> >> roundtrip as with resize window.
>> >
>> > Resize is actually a huge deal. It is very very heavy. At least I
>> > do not want to incur the same penalty to moving, now that using a
>> > compositor has finally made moving a very cheap operation.
>> >
>> >> Change i propose are not big, it's a fairly small contained change, i
>> >> will try to hack something over the weekend.
>> >
>> > This discussion is has lead me to conclude, that it is
>> > near-impossible to reliably support and present sub-pixel rendering
>> > in general without telling clients where their surfaces are on an
>> > output.
>> >
>> > Not telling clients where their surfaces are has been a major
>> > design decision in Wayland this far, so it seems Wayland is somewhat
>> > incompatible with the concept of sub-pixel rendering. Feasible
>> > cases are limited to just some special cases, where the output
>> > device has a translation-invariant pixel pattern, and the
>> > compositor does not apply any other transformations than
>> > translation.
>> >
>> > With that in mind, I think it's ok to enable sub-pixel rendering
>> > where it is feasible, but I would not expose surface positions or
>> > transformations to clients for it.
>> >
>> > Therefore I'd simply propose for compositors to advertise outputs
>> > with sub-pixel order as unknown or none, unless the output is known
>> > to be translation invariant and surfaces are only translated.
>> >
>> >
>> > Thanks,
>> > pq
>>
>> I still believe that my proposal is totally doable and does not break
>> wayland paradigm. I should stress that the compositor (wayland server)
>> is always in charge that means that the compositor can decide to not
>> ping pong with the application while moving the window but only update
>> the transformation matrix and ask for a redraw from client once the
>> window no longer move.
>
> This introduces temporary glitches, which we work hard to eliminate.
> Unless you mean window outline moves instead of window content?
>
> (Window outline resizes are actually something I'd personally like
> to see, would make Weston on Raspberry Pi a lot more smooth
> experience.)

Yes temporary glitch but better rendering for the common case where
window does not move.

>> Window being move or resized is not typical usage of desktop, by that
>> i means user is not going to constantly move its window or resize
>> them, it happens seldomly during a session. That being said i stress
>> again that the compositor is in charge and if it want each frame
>> sub-pixel perfect it could ask for a redraw by the application, or it
>> could just wait for the window to reach its final position as attended
>> by the user and then send the matrix and ask for new sub-pixel perfect
>> content.
>>
>> You can get both, autonomous compositor window move (with no client
>> communication) and sub-pixel perfect rendering on non moving window.
>
> Yeah, with glitches, if the monitor sub-pixel pattern is not
> translation invariant.

No you can do proper rendering if you know the sub-pixel pattern and
the transformation matrix.

>
>> My point here is that i would hate for an api that just encompass
>> special cases, while transmitting the transformation matrix as long as
>> providing per output dpi/geometry informations would allow to handle
>> any case we could ever face.
>
> How would you transmit transformations that are not representable
> by a matrix? Nothing says we are limited to matrices, that is also
> just a special case. Or would you introduce that limitation in the
> protocol?

What kind of transformation are not representable by matrix, i am
really curious on that. Unless you want to break the window into tile
and show tile at different position which would be very weird. But
still doable just have the server send an identify matrix.

> In my opinion, the whole idea of clients rendering in screen space
> goes against a compositing architecture. Once you let clients to draw
> in screen space, the compositor is no longer free to move and
> transform the image. Even blending might break the image. The only
> thing I can think of what the compositor can do without asking and
> waiting for a new image from the client is re-stacking. Either
> that, or temporarily produce wrong output.

As i sais compositor is still in charge and nothing i propose to add
forbid or change what compositor can do. Will there be a slight
difference in rendering btw non moving/changing/resize/... window and
stable one, depend on the compositor it can either request redraw
supixel perfect for each step of the animation or leave with the fact
of unperfect result during animation and it won't be terribly wrong
output you make it sounds like it will be horrible.

>
> Formulating your idea as an optional extension would be ok, and
> maybe that is your plan.
>
> Or maybe we should just propose to remove all sub-pixel features from
> the core protocol? Well, not remove, but add a note that the
> sub-pixel information is not useful without further extensions.

I think the sub-pixel information is just too limited but still usefull.

>
> About dpi. There is going to be a clipping and scaling extension to
> allow taking advantage of hardware scalers, and that will be an
> orthogonal companion extension to sub-surfaces.
>
> We don't have any proposal of the clipping & scaling extension, but
> writing it in terms of DPI does not sound right to me. It probably will
> somehow relay the intended surface size separately from the buffer
> size, and also allow to use just a sub-rectangle of the buffer contents.
> These would be sizes in integer pixels, so we avoid any rounding
> problems from multiplying fixed or floating point numbers.
>
> Helping with that would be very useful. The idea is to have an
> extension interface to wl_surface, just like wl_shell_surface and
> wl_subsurface are.
>
>
> Thanks,
> pq

On DPI side i only proposed that client tell the compositor the DPI at
which it rendered its buffer so that compositor can upscale on high
dpi display.

Cheers,
Jerome