[PATCH 0/2] Support for high DPI outputs via scaling

[Jason Ekstrand]

On Mon, May 13, 2013 at 9:54 AM, Alexander Larsson <alexl at redhat.com> wrote:

>
> On mån, 2013-05-13 at 14:40 +0200, John Kåre Alsaker wrote:
>
> >
> >         I don't think this will work in practice. I know for sure that
> >         e.g. Gtk
> >         is not set up to do any reasonable non-integer scaling. It
> >         will just
> >         scale up all drawing by a fractional factor without any
> >         rounding
> >         anywhere, causing everything to become fuzzy. We will
> >         eventually have
> >         alternative icons for higher resolutions, but these will be at
> >         2x scale,
> >         not at generic fractional factor (that is not really doable
> >         without
> >         using pure-vector icons with some complex hinting method).
> >         Although, I
> >         guess that in the case of text the scaling will be ok, so for
> >         some
> >         usecases it might be OK.
> > It will work very well for things designed to be scalable, browsers
> > are an example. GTK could just fall back to integer scaling.
>
> Browsers are not really scalable like that, css page layout is generally
> based on
> the CSS "Px" definition, and per e.g.
> http://static.zealous-studios.co.uk/projects/web_tests/PPI%20tests.html:
>
>         For lower-resolution devices [i.e. non-print], and devices with
> unusual
>         viewing distances, it is recommended instead that the anchor unit
> be the
>         pixel unit. For such devices it is recommended that the pixel unit
> refer
>         to the whole number of device pixels that best approximates the
>         reference pixel.
>
> I.e. "whole number of pixels" => scale by integer matches best what CSS
> wants.
>
> >         So, it's my opinion that supporting fractional scaling is an
> >         unnecessary
> >         burden on compositors for something that is not very useful in
> >         practice.
> >         Thats just my opinion though, and the proposal as such can
> >         handle
> >         fractional scaling fine by just changing the scale factors to
> >         fixed
> >         type.
> > It is of no burden on compositors at all, only for clients which
> > choose to implement it.
>
> It is, as long as clients are allowed to specify a fractional scaling
> factors compositors need to be able to handle scaling by that (for
> instance if the window appears partially on a non-highres monitor.
>

While this isn't a huge problem on GL-based compositors it will cause a
problem for software compositors.  Any scaling for that matter is a
potential problem there.  However, a factor of two or something shouldn't
be too bad in software.


>  >         > In mirrored/clone mode only a single wl_output would be
> >         presented to
> >         > clients with a single scale factor, so priorities won't
> >         matter in that
> >         > case.
> >
> >
> >         That is not necessarily so. We might very well want to know
> >         that there
> >         are two displays, with say different RGB subpixel order, etc.
> > The compositor could unify this information into a single wl_output.
>
> And what would it then report for physical width/height, subpixel, make,
> model, etc? Seems pretty weird.
>
> > I suggest we send 3 scaling factors to clients. A lower bound where
> > factors lower will be scaled up. The desired scaling factor. A upper
> > bound where factors above will be scaled down. Clients should find the
> > scaling factor in lower bound to upper bound they are able to render
> > at which is closes to the desired factor. Failing that they should
> > find the first factor they are able to render at above the upper
> > bound. When displayed on multiple monitors it could try to remain
> > sharp on as many monitors as possible.
>
> I don't quite understand this. Any factor lower than the "desired"
> factor has to be scaled up, no?
>
> Lets take a concrete example. The macbook pro 15" has a 2880 x 1800
> native panel. This is "normally" driven as 1440 x 900 with a scale
> factor of 2. On OSX you can also select a 1920x1200 resolution, which is
> nominally a scale factor of 1.5. However, on OSX what they do is that
> apps render to a framebuffer at twice the resolution (3840x2400) and its
> then scaled it down to 2880x1800 to make it look reasonable.
>

Actually, this seems like a fairly reasonable way for the client to
render.  It can simply render at a sufficiently high integer multiple of
the resolution it wants to look really good on such that it's always
downscaling.  Or, if it's worried about memory, let the compositor
upscale.  However, the key here is that the scale factor is an integer
multiple of *one* of the monitor scale factors.  Whether or not the monitor
scale factor is an integer doesn't matter for this.


> On such a setup in wayland the wl_output would be 1920x1200, with a
> scaling factor of 1.5. For a 800x601 window a hidpi applications could
> then either pick a buffer of 1600x1202 buffer and set a scaling factor
> of 2 causing the compositor to downscale it (similar to OSX above), or
> it could create a buffer at 1200x902 (nearest to the true 901.5) and and
> set scaling factor to 902/600 (i.e. 1.50333..) and have the compositor
> not scale it (which is a minor issue due to the 1.5 != 1.50333..).
>
> In this case the desired scaling factor would be 1.5. What do you expect
> the upper and lower bounds to be?
>

I don't know if the upper/lower limits is the solution, but this is
certainly an issue.  In a case like that, I would say that the application
simply works in the coordinates of the "prefered" output.  This includes
input events etc.  This may mean that the size of the window in points
isn't an integer.  If the toolkit doesn't want to deal with that, they can
come up with a solution (maybe just require a multiple of 2 for the size in
this case).

Allow me to attempt to restate something I tried to point out in an earlier
e-mail (now that I understand the problem better).  As a disclaimer, while
I have worked with a number of toolkits from the client-side, I am not a
toolkit developer.  Feel free to tell me when there are hidden issues that
have to do with toolkit innards (but please be specific so I can understand
the exact problem).

One line that I think needs to be drawn more distinctly here is what
happens at the application level, the toolkit level, and the compositor
level.  At the application level, we would rather they not even notice the
change unless the want to.  While it might be convenient at times to know
this information in a GTK or Qt app, apps should work nicely on hidpi
displays without any modification.  They will probably need
higher-resolution or multi-resolution icons, but other than that they
should "just work".

Where to separate the toolkit from the compositor becomes more of an
interesting issue.  The only place I have seen multiple densities handled
well is Android (Windows 8 does something, but they're using HTML5/CSS so
they can cheat a bit).  On Android, they have a concept of "display pixels"
or dp which basically corresponds to "points" in your above proposal.  When
you lay out an android UI, you do so in display pixels and then the toolkit
automatically handles scaling everything.  Android basically has 4
different "scale factors": 0.75, 1.0, 1.5, 2.0 (I think there's a 3.0 for
some of these crazy 450 DPI phones).  The reason why Android's system works
so well (and looks so good) is that all the scaling is done at the toolkit
level and they don't really have to bother with multiple outputs.

Unfortunately, we do have to bother with multiple outputs.  We all know how
apple solved this and it assumes that there will only ever be two screen
densities in existence.  Since apple controls their hardware, they could
assume this.  We cant.  I think what I would suggest is a hybrid
toolkit/compositor approach as follows.

1. Apps simply define everything in points with the understanding that it
may not directly correspond to pixels on the display.  For icons and other
images, they can provide multiple versions or use vector graphics or
something.

2. The toolkit picks a "nice" scaling factor to go between pixels on its
preferred output to points for the client.  By "nice" I mean that you'd
rather scale by 1.5 than, say, 1.587.  It then renders at the resolution of
the preferred output (or possibly an integer multiple so other outputs
downscale instead of upscale).  When it hands the buffer to the compositor,
it gives the compositor a (probably integer) scale factor relative to a
particular output.  Yes, this means the toolkit may have to ceiling or
floor some values to make everything fit nicely.  However, the point is
that the scale factor presented to the user doesn't have to match what
happens in the toolkit.

3. The compositor then takes the image provided by the toolkit and scales
it for all of the outputs.  On the toolkit's preferred output, it shouldn't
scale at all (or possibly an integer multiple).  On the other outputs, it
would scale based on how the outputs are scaled relative to each other and
the provided scale factor.  Whether the compositor sends events in pixels
or points probably doesn't matter that much (up to rounding error).
However, I think I would vote for pixels on the client's preferred monitor
so that pixel-perfect clicking is preserved.

There are a couple of other things I think are worth mentioning here as
well.  First, is that the client can't know how much of it is on which
surface.  There is nothing in the protocol (a this point) to indicate that
beyond which outputs it is on.  Surfaces don't know their absolute
positions.

Second, we are going to have to deal with fractional scaling even if we
restrict to integers.  Consider the following example.  Say I have two
monitors: A and B which are at 200 DPI, and 300 DPI respectively.  An
application has a number of choices for resolution:

 - It could render at 200 DPI and then scale by 3/2 for B.
 - It could render at 300 DPI and scale by 3/2 for A.
 - It could render at 100 DPI and scale by 2 for A and 3 for B and look bad
on both
 - It could render at 400 DPI and scale by 3/4 for B and 1/2 for A

and I'm sure you could come up with more.  However, the point is that
unless we are going to make the extremely restrictive assumption that
monitors come in powers of two, we cannot get around fractional scaling.

Thanks,
--Jason Ekstrand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20130513/41c0c21c/attachment-0001.html>