<div dir="ltr">On Mon, May 13, 2013 at 8:52 PM, Jason Ekstrand <span dir="ltr"><<a href="mailto:jason@jlekstrand.net" target="_blank">jason@jlekstrand.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5">On Mon, May 13, 2013 at 9:54 AM, Alexander Larsson <span dir="ltr"><<a href="mailto:alexl@redhat.com" target="_blank">alexl@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><br>
On mån, 2013-05-13 at 14:40 +0200, John Kåre Alsaker wrote:<br>
<br>
><br>
> I don't think this will work in practice. I know for sure that<br>
> e.g. Gtk<br>
> is not set up to do any reasonable non-integer scaling. It<br>
> will just<br>
> scale up all drawing by a fractional factor without any<br>
> rounding<br>
> anywhere, causing everything to become fuzzy. We will<br>
> eventually have<br>
> alternative icons for higher resolutions, but these will be at<br>
> 2x scale,<br>
> not at generic fractional factor (that is not really doable<br>
> without<br>
> using pure-vector icons with some complex hinting method).<br>
> Although, I<br>
> guess that in the case of text the scaling will be ok, so for<br>
> some<br>
> usecases it might be OK.<br>
> It will work very well for things designed to be scalable, browsers<br>
> are an example. GTK could just fall back to integer scaling.<br>
<br>
</div>Browsers are not really scalable like that, css page layout is generally based on<br>
the CSS "Px" definition, and per e.g.<br>
<a href="http://static.zealous-studios.co.uk/projects/web_tests/PPI%20tests.html" target="_blank">http://static.zealous-studios.co.uk/projects/web_tests/PPI%20tests.html</a>:<br>
<br>
For lower-resolution devices [i.e. non-print], and devices with unusual<br>
viewing distances, it is recommended instead that the anchor unit be the<br>
pixel unit. For such devices it is recommended that the pixel unit refer<br>
to the whole number of device pixels that best approximates the<br>
reference pixel.<br>
<br>
I.e. "whole number of pixels" => scale by integer matches best what CSS wants.<br></blockquote></div></div></div></div></div></blockquote><div>My point was that browsers can scale up websites by fractional amounts even though it will look somewhat better with integer factors.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><br>
> So, it's my opinion that supporting fractional scaling is an<br>
> unnecessary<br>
> burden on compositors for something that is not very useful in<br>
> practice.<br>
> Thats just my opinion though, and the proposal as such can<br>
> handle<br>
> fractional scaling fine by just changing the scale factors to<br>
> fixed<br>
> type.<br>
> It is of no burden on compositors at all, only for clients which<br>
> choose to implement it.<br>
<br>
</div>It is, as long as clients are allowed to specify a fractional scaling<br>
factors compositors need to be able to handle scaling by that (for<br>
instance if the window appears partially on a non-highres monitor.<br></blockquote><div><br></div></div></div><div>While this isn't a huge problem on GL-based compositors it will cause a problem for software compositors. Any scaling for that matter is a potential problem there. However, a factor of two or something shouldn't be too bad in software.<br>
</div></div></div></div></blockquote><div>Compositors do not have to scale anything at all. Even with fractional scaling factors a compositor could limit itself to just scaling with integer factors. I don't think software compositor is a case worth considering either. Nobody wants to have those, especially with high-DPI displays.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>
</div><div><div class="h5"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
> > In mirrored/clone mode only a single wl_output would be<br>
> presented to<br>
> > clients with a single scale factor, so priorities won't<br>
> matter in that<br>
> > case.<br>
><br>
><br>
> That is not necessarily so. We might very well want to know<br>
> that there<br>
> are two displays, with say different RGB subpixel order, etc.<br>
> The compositor could unify this information into a single wl_output.<br>
<br>
</div>And what would it then report for physical width/height, subpixel, make,<br>
model, etc? Seems pretty weird.<br></blockquote></div></div></div></div></div></blockquote><div>Even if two wl_outputs would be presented, they would have the same resolution and scaling factor so priorities still won't matter.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><br>
> I suggest we send 3 scaling factors to clients. A lower bound where<br>
> factors lower will be scaled up. The desired scaling factor. A upper<br>
> bound where factors above will be scaled down. Clients should find the<br>
> scaling factor in lower bound to upper bound they are able to render<br>
> at which is closes to the desired factor. Failing that they should<br>
> find the first factor they are able to render at above the upper<br>
> bound. When displayed on multiple monitors it could try to remain<br>
> sharp on as many monitors as possible.<br>
<br>
</div>I don't quite understand this. Any factor lower than the "desired"<br>
factor has to be scaled up, no?<br></blockquote></div></div></div></div></div></blockquote><div>I expect a compositor to render deviations of the desired scaling factor without scaling windows. The range when this is allowed is reported to clients so they can try to render at a size which will avoid scaling.</div>
<div><br></div><div>For example a compositor may want to use a 1-1.2 range with 1.1 as the desired scaling factor. A clients which are only able to draw at integer scaling factor would round that up to 2 and let the compositor downscale it. When the range for which compositor won't scale is send to clients we can avoid this.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Lets take a concrete example. The macbook pro 15" has a 2880 x 1800<br>
native panel. This is "normally" driven as 1440 x 900 with a scale<br>
factor of 2. On OSX you can also select a 1920x1200 resolution, which is<br>
nominally a scale factor of 1.5. However, on OSX what they do is that<br>
apps render to a framebuffer at twice the resolution (3840x2400) and its<br>
then scaled it down to 2880x1800 to make it look reasonable.<br></blockquote></div></div></div></div></div></blockquote><div>Rendering at 3840x2400 then scaling down to 2880x1800 is bad for performance and makes the result unsharp. It is much preferable that clients render into 2880x1800 directly with a 1.5 scaling factor (for clients that are capable of doing that).</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5"><div><br></div></div>
</div><div>Actually, this seems like a fairly reasonable way for the client to render. It can simply render at a sufficiently high integer multiple of the resolution it wants to look really good on such that it's always downscaling. Or, if it's worried about memory, let the compositor upscale. However, the key here is that the scale factor is an integer multiple of *one* of the monitor scale factors. Whether or not the monitor scale factor is an integer doesn't matter for this.<br>
</div></div></div></div></blockquote><div>We can't really downscale with integer multipliers, just upscale. If we downscale, rendering at a (much) higher resolution doesn't really help much. We want to avoid upscaling though.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>
</div><div class="im"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On such a setup in wayland the wl_output would be 1920x1200, with a<br>
scaling factor of 1.5. For a 800x601 window a hidpi applications could<br>
then either pick a buffer of 1600x1202 buffer and set a scaling factor<br>
of 2 causing the compositor to downscale it (similar to OSX above), or<br>
it could create a buffer at 1200x902 (nearest to the true 901.5) and and<br>
set scaling factor to 902/600 (i.e. 1.50333..) and have the compositor<br>
not scale it (which is a minor issue due to the 1.5 != 1.50333..).<br>
<br>
In this case the desired scaling factor would be 1.5. What do you expect<br>
the upper and lower bounds to be?<br></blockquote></div></div></div></div></blockquote><div>If the compositor allows deviations in area of 10%, I'd expect the lower and upper bounds to be 1.42 ((1.5<span lang="NO-NYN" style="font-size:11.0pt;line-height:115%;font-family:"Calibri","sans-serif"">²</span> * 0.9)^<span lang="NO-NYN">½</span>) and 1.57
((1.5<span lang="NO-NYN" style="font-size:11.0pt;line-height:115%;font-family:"Calibri","sans-serif"">²</span> * 1.1)^<span lang="NO-NYN">½</span>) respectively.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
</blockquote></div></div><br></div><div class="gmail_extra">I don't know if the upper/lower limits is the solution, but this is certainly an issue. In a case like that, I would say that the application simply works in the coordinates of the "prefered" output. This includes input events etc. This may mean that the size of the window in points isn't an integer. If the toolkit doesn't want to deal with that, they can come up with a solution (maybe just require a multiple of 2 for the size in this case).<br>
</div></div></blockquote><div>The size of a window should always be in pixels at the protocol level.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra">
<br></div><div class="gmail_extra">Allow me to attempt to restate something I tried to point out in an earlier e-mail (now that I understand the problem better). As a disclaimer, while I have worked with a number of toolkits from the client-side, I am not a toolkit developer. Feel free to tell me when there are hidden issues that have to do with toolkit innards (but please be specific so I can understand the exact problem). <br>
<br>One line that I think needs to be drawn more distinctly here is what happens at the application level, the toolkit level, and the compositor level. At the application level, we would rather they not even notice the change unless the want to. While it might be convenient at times to know this information in a GTK or Qt app, apps should work nicely on hidpi displays without any modification. They will probably need higher-resolution or multi-resolution icons, but other than that they should "just work".<br>
<br></div><div class="gmail_extra">Where to separate the toolkit from the compositor becomes more of an interesting issue. The only place I have seen multiple densities handled well is Android (Windows 8 does something, but they're using HTML5/CSS so they can cheat a bit). On Android, they have a concept of "display pixels" or dp which basically corresponds to "points" in your above proposal. When you lay out an android UI, you do so in display pixels and then the toolkit automatically handles scaling everything. Android basically has 4 different "scale factors": 0.75, 1.0, 1.5, 2.0 (I think there's a 3.0 for some of these crazy 450 DPI phones). The reason why Android's system works so well (and looks so good) is that all the scaling is done at the toolkit level and they don't really have to bother with multiple outputs.<br>
<br></div><div class="gmail_extra">Unfortunately, we do have to bother with multiple outputs. We all know how apple solved this and it assumes that there will only ever be two screen densities in existence. Since apple controls their hardware, they could assume this. We cant. I think what I would suggest is a hybrid toolkit/compositor approach as follows.<br>
<br></div><div class="gmail_extra">1. Apps simply define everything in points with the understanding that it may not directly correspond to pixels on the display. For icons and other images, they can provide multiple versions or use vector graphics or something.<br>
<br></div><div class="gmail_extra">2. The toolkit picks a "nice" scaling factor to go between pixels on its preferred output to points for the client. By "nice" I mean that you'd rather scale by 1.5 than, say, 1.587. It then renders at the resolution of the preferred output (or possibly an integer multiple so other outputs downscale instead of upscale). When it hands the buffer to the compositor, it gives the compositor a (probably integer) scale factor relative to a particular output. Yes, this means the toolkit may have to ceiling or floor some values to make everything fit nicely. However, the point is that the scale factor presented to the user doesn't have to match what happens in the toolkit.<br>
<br></div><div class="gmail_extra">3. The compositor then takes the image provided by the toolkit and scales it for all of the outputs. On the toolkit's preferred output, it shouldn't scale at all (or possibly an integer multiple). On the other outputs, it would scale based on how the outputs are scaled relative to each other and the provided scale factor. Whether the compositor sends events in pixels or points probably doesn't matter that much (up to rounding error). However, I think I would vote for pixels on the client's preferred monitor so that pixel-perfect clicking is preserved.<br>
<br></div><div class="gmail_extra">There are a couple of other things I think are worth mentioning here as well. First, is that the client can't know how much of it is on which surface. There is nothing in the protocol (a this point) to indicate that beyond which outputs it is on. Surfaces don't know their absolute positions.<br>
<br></div><div class="gmail_extra">Second, we are going to have to deal with fractional scaling even if we restrict to integers. Consider the following example. Say I have two monitors: A and B which are at 200 DPI, and 300 DPI respectively. An application has a number of choices for resolution:<br>
<br> - It could render at 200 DPI and then scale by 3/2 for B.<br></div><div class="gmail_extra"> - It could render at 300 DPI and scale by 3/2 for A.<br></div><div class="gmail_extra"> - It could render at 100 DPI and scale by 2 for A and 3 for B and look bad on both<br>
</div><div class="gmail_extra"> - It could render at 400 DPI and scale by 3/4 for B and 1/2 for A<br><br></div><div class="gmail_extra">and I'm sure you could come up with more. However, the point is that unless we are going to make the extremely restrictive assumption that monitors come in powers of two, we cannot get around fractional scaling.<br>
</div></div></blockquote><div>Yes, we do need fractional scaling even without fractional scaling factor to give a good experience.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra">
<br></div><div class="gmail_extra">Thanks,<br></div><div class="gmail_extra">--Jason Ekstrand<br></div></div>
</blockquote><br></div>