surface buffer cardinality and outputs

[Jason Ekstrand]

On Wed, Mar 13, 2013 at 10:48 AM, Sylvain BERTRAND <sylware at legeek.net> wrote:
> Hi,
>
> The protocol does allow several outputs in the compositor space
> but does not allow a client to render a surface based on output
> properties (dpi,subpixel,frame buffer pixel format...) since
> you can attach only one buffer to a surface.
>
> If wayland wants to give a client the ability to render its
> surfaces in regards of the properties of the outputs, the
> protocol should be modified to take into account that a client
> may have to render a surface differently for each output. It may
> have a "don't care about output properties" mode which would
> force the use of one buffer for all outputs.
>
> On the other hand, if wayland does not want to let the client
> render a buffer specific to the properties of each output, the
> output object should not expose any of its properties to clients.
> In another words, to give the subpixel format of an output to
> client and not supporting rendering per output would be
> inconsistent.
>
> How does we cleanly fix this, or did I miss something?

Sylvain,
I think another question, probably equally important, is how should
toolkits handle multiple output properties?  I know that wayland isn't
supposed to be dictating toolkits, but we need to think about that
angle as we look at the protocol.

Many toolkits may want to avoid the question as much as possible
because rendering multiple times is expensive. However, for the sake
of the question, let's assume your toolkit does want to adapt to
screen properties.  Things like subpixel format are fairly simple:
just render two copies one in each format.  How about trying to
respond to different display densities?  In this case you have two
different buffers that each represent the surface but they are
different resolutions, how big is the surface?

Apple chose to fix this problem by simply making their retina displays
exactly 4 times the number of pixes (double in both directions) of the
corresponding regular display.  This makes everything easy because you
just render at a higher resolution and can easily scale down.  In
Android, they have a unit called a "display pixel" that is supposed to
be DPI agnostic.  Depending on the phone, 1dp may be 0.5px, 1px,
1.5px, or 2px.  Then, if you specify everything in dp, things scale
nicely (you still have to handle different image resolutions).

While we're trying to tackle this question, I think there's another
use-case we need to consider.  What if I want to have a buffer that is
a different resolution than the corresponding surface and have the
compositor scale it.  We are already doing this for fullscreen but
cannot, at present, do it on an arbitrary surface.  Initially this
sounds like a bad idea because widgets and decorations would be
blurry.  But this all changes with pq's subsurfaces.  Let's say I have
a video player that is playing a 720x480 video stream from a DVD.  If
I am using a subsurface to hold the video, it would be really nice if
I could keep the buffer for the video at 720x480 and simply have the
compositor scale the video as it composites.

Ok, now that I'm done throwing out questions, I have an idea that
*may* help answer some of them.  What if, instead of changing
wl_surface, we add some new buffer types.  One example would be a
buffer that was actually a collection of "child" buffers keyed to
output properties.  This would be similar to mipmaps in OpenGL
textures.  Then when the compositor goes to draw the surface it would
select the best "child" buffer and use that one to render the surface.
 Another type of buffer would be one that wraps another buffer (egl,
or shm) and in tells the compositor to ignore the backing buffer's
size and scale it to the size of the wrapper.

There's my thoughts. Thanks,
--Jason Ekstrand