[RFC PATCH v2] Add xdg-output protocol

[Jonas Ådahl]

This E-mail is quite long, but I tried to reply to some parts.

On Wed, Jul 12, 2017 at 12:07:29PM +0300, Pekka Paalanen wrote:
> On Fri, 7 Jul 2017 04:21:57 -0400 (EDT)
> Olivier Fourdan <ofourdan at redhat.com> wrote:
> 
> > Hi Pekka,
> > 
> > > it's very hard for me to wrap my head around this, so the below may
> > > sound a bit harsh, sorry. I don't mean to rant, but I feel there is
> > > something fundamental amiss. I am diving back into the high-level
> > > design which is fairly separated from the xdg_output interface.  
> > 
> > No worries, but my goal with the protocol proposal (being an Xwayland
> > specific proposal initially, or merely an additional wl_output event
> > or now a more generic xdg_output protocol to be extended in the
> > future with whatever people want for desktop output uses) is to make
> > Xwayland work with the current existing design/implementations, not
> > to advocate for or against an existing design or particular
> > compositor implementation.
> 
> Hi Olivier,
> 
> if the intention is not to evaluate whether the current implementations
> are actually workable in the long run, then should the new interface be
> named so that we can throw it away if it actually doesn't work? And not
> design anything else directly on top of it, so that it remains
> relatively easy to throw away, i.e. stop using?
> 
> > 
> > WRT fractional scaling, my understanding of the design in mutter is
> > based on this document:
> > 
> > https://mail.gnome.org/archives/gnome-shell-list/2017-June/msg00000.html
> > 
> > I cannot really comment on the design decisions, nor how those
> > decisions were made, Jonas would probably be in a better position for
> > this.
> 
> That email talks about a new feature, separated coordinate spaces:
> logical and physical. I believe that is the correct design.
> 
> But I have understood that you want to make Xwayland work with the
> design before that one, which I cannot see how it could work. Do you
> really need to make Xwayland work well with that old design?

The benefit of the current/old design (in mutter) is that X11 clients
don't particularly regress in functionality, compared to how it worked
before. HiDPI aware clients work just as well, and non-HiDPI clients
works just as bad.

Naturally, if we can make both HiDPI aware and HiDPI unaware X11 clients
work good, that's the most optimal, but I find it hard to believe we can
make that happen without adaptations to HiDPI aware X11 clients (like
mentioned EWMH extensions or something).

> 
> I fully agree with section 1, Overall approach.
> 
> Section 5 mentions a hwdb-alike database but keyed by EDID data. This is
> something Keith Packard needs for his VR work to identify HMDs and I
> discussed with him about it a bit. I think the scaling info would be a
> good fit there.
> 
> Section 7 touches the game and fullscreen topic, which I believe is
> important for deciding how RandR should advertise output resolutions.
> 
> Section 8 is exactly what we are talking about in this thread, right?

Right. Lets split up Xwayland support into three distinguishable parts:

(A) Plain old Xwayland support with no HiDPI awareness or anything

(B) Fullscreen windows with buffer size matching the monitor framebuffer
it is fullscreen on

(C) HiDPI aware X11 clients.


For (A), this protocol solves the problem more or less completely
without any further changes. No input transformation changes needed
anywhere, not in Xwayland, not in any compositor. It is just a way to
communicate the logical layout of the screen to Xwayland so that it can
position its buffer-scale-1 windows in a grid that matches the global
compositor coordinate space.

> 
> 
> What resolution do you want RandR to advertise to X11 apps?
> 
> If it is the physical output resolution, the window size in logical
> units will be wrong unless XWM can reliably detect the X11 application
> is actually intending to use the physical resolution. How would one do
> that?

This is about part (B).

How? I'd say, probably only using "best effort" emulation strategies,
like detecting fullscreen windows matching the physical resolution set
and assuming anything else should be hidden, then using wp_viewporter or
something to make it to the logical size, which should result in no
scaling being done by the compositor.

> 
> If it is the logical output size, the window size in logical units will
> be correct, but the X11 application cannot provide a buffer of the
> physical output resolution, which means the Wayland compositor must
> scale the image. If the buffer can hit direct scanout, that should be
> no problem, but otherwise it could be a performance issue. (I agree to
> prefer this option.)

It'll be a regression quality wise though, as something somewhere will
have to scale up from the logical to the physical resolution. Long term,
this is not acceptable as it'll mean games and other applications where
graphics quality is important that also won't be ported would regress.

However, as a first step I think this is what we should do since it's
trivial. This is what (A) does.

> 
> In any case, there is a single (default) "scaling factor" covering all
> X11 clients in an Xwayland instance, because neither Xwayland nor XWM
> have per-window knowledge of the scaling the X11 app used. Also, X11
> apps are laid out in a global coordinate space spanning all outputs
> (within a X11 SCREEN), and also input works in that global coordinate
> space with trivial conversions to per-window coordinates. I assume
> Xwayland converts per-wl_surface input coordinates to global input
> coordinates in the X11 space before dispatching to the X11 windows,
> which means that the Wayland server must send input events taking that
> into account. How will input work in the proposals?
> 
> I do not like the idea of special-casing the Xwayland Wayland client in
> the Wayland compositor outside of XWM, even less if it requires special
> mangling for input as well. How much special casing do you do? Or maybe
> I'm just biased and it would actually be fairly easy to implement in
> Weston with a little bit of API... Quentin might have some opinions
> here.

Are you talking about (C) here? I don't really have a thought out idea
or plan how it should work, but EWMH is mentioned, but whether Xwayland
should play a part or not I'm not sure. I suspect it'll require changes
to X11 clients to make it work though.

> 
> > Regarding this particular discussion about the need of xdg_output and
> > logical size/position, I am completely open to suggestions, how would
> > see Xwayland work with both weston and mutter as they are now and
> > mutter in the future when it implements fractional scaling, without
> > adding logical size in xdg_output (or even not adding xdg_output at
> > all)?
> 
> I wouldn't say "without", I just do not understand how it all is
> supposed to work in a mixed-dpi hardware setup. If we design an
> interface that just cannot work in a mixed-dpi setup, we should be
> able to throw it away later.
> 
> So I'm starting to think that the way forward is to design the new
> interface such that it is easy to deprecate if it turns out to be a
> dead end, or keep it if turns out future-proof.

Well, this is an unstable wayland-protocols protocol, so we are
completely free to abandon it right? If we need xdg_output to do
something else after having abandoned the purpose of it now, we're also
free to reinvent it.

> 
> Yeah, I'm going in circles here as I'm trying to find out what the
> goals are.
> 
> 
> > 
> > Mixed dpi doesn't work in GNOME/x11, does it?
> 
> I guess - do you want to keep that status quo?

It does not indeed.

> 
> > GNOME implementation of HiDPI on x11 uses an xsettings, which by
> > definition is per screen (in x11 terminology), thus not per monitor.
> > 
> >   https://wiki.gnome.org/HowDoI/HiDpi
> > 
> > So fixing mixed DPI in Xwayland will require additional mechanisms
> > (possibly new EWMH properties?), but this is not part of this
> > proposal.
> 
> I would not want to go there. I would prefer to keep all of X11 in a
> single scale factor, and scale accordingly in the compositor. That is
> what I call a working mixed-dpi setup - it may not be pixel-perfect
> pretty, but at least everything is readable and not blown up.
> 
> Maybe the definition of "mixed-dpi support" is what we disagree here.
> 
> In my mind, Xwayland cannot provide full mixed-dpi support, because X11
> is simply incapable of it with current toolkits. It can provide partial
> mixed-dpi support however, where the Wayland compositor is
> automatically scaling, assuming all X11 apps are configured to use the
> same scale factor.
> 
> It same as with Wayland apps: HiDPI unaware apps get partial mixed-dpi
> support, while HiDPI aware apps get the full mixed-dpi support.
> 
> This would classify all HiDPI-aware X11 apps as being somewhere between
> HiDPI unaware and aware Wayland apps: they cannot communicate their
> scaling factor, but they can be configured as a group to always draw
> with a given scaling factor.
> 
> > However, nothing would stop an x11 client or toolkit from querying
> > the monitor's mode and physical size (using xrandx), computing the
> > current DPI depending on the monitor it resides and adapting its
> > rendering based on that. X11 has all the mechanisms in place for
> > that, it's just that most clients won't do that (iirc, Firefox has
> > something like that at some point, not sure it's still used though)
> 
> That's actually a case I didn't even think of. I've only been thinking
> about apps that want to make their window cover an output, like games.
> This is a very good topic to bring up.
> 
> > That makes a wide range of different x11 clients who behave
> > differently:
> > 
> >  - Plain old x11 clients, who don't know anything about DPI
> >  - gtk+/GNOME clients, who base their rendering on a specific
> > xsettings Gdk/WindowScalingFactor and/or a envvar read by clutter,
> >  - Some other app trying to compute DPI themselves (very few, I think
> > of Firefox, LibreOffice maybe? not sure at all about those)
> > 
> > I'm don't see how Xwayland could change its behavior for each app,
> > xrandr is not per client. I guess we could come up with a new window
> > property that Xwayland could monitor and set the buffer scale
> > accordingly, maybe? Anyhow, that's going off topic wrt the xdg_output
> > protocol, I'm afraid.
> 
> I agree, I don't see the worth in implementing per-X11-window scales.
> 
> Therefore we need to stick with one scale over all X11 clients. If
> there are plain old clients in that set, then the only workable factor
> for X11 is scale=1. Would you agree with that?
> 
> > The "benefit" I see in "lying" to the clients in xrandr by
> > advertising a lower mode than actual is that it solves the problem in
> > a consistent way for all these cases even those clients who try to
> > compute the DPI themselves).
> 
> How does it solve it for all the cases? Does it not help only with the
> third case, while the first and second are unaffected? Not to belittle
> the benefit, just wanting to be accurate.

For the first case, this proposal makes it at least work at all, i.e.
xterm becomes readable by default.

For the second case, in GNOME, we (at least now) simply set the Xsetting
scaling factor to 1 via gnome-settings-daemon.

For the third case, as Olivier mentiones, makes self calculating clients
think the resolution isn't that high so they might avoid scaling up.

> 
> > There are some additional discussions in
> > https://bugs.freedesktop.org/show_bug.cgi?id=93315
> 
> > Xwayland computes the screen size by adding up the output size and
> > position. So it all comes from the same wl_output description (for
> > now). 
> > 
> > Plain x11 screen width/height, Xrandr, Xinerama and even Xvidmode
> > will remain consistent. 
> 
> That's nice, it's all consistent. We just need to be similarly
> consistent in what we advertise as size in RandR. It means it has to
> make sense in a mixed-dpi setup.
> 
> 
> 
> > If randr advertise the output mode 3840×2160 as 2560×1620 with a
> > fractional scale of 1.5, then the compositor would scale the buffer
> > up by that fractional scaling factor, i.e. 1.5, which gives the
> > expected 3840×2160.
> > 
> > I don't see how downscaling an already tiny x11 window would work.
> > Take a basic x11 app, say xterm, with its default fixed font size. It
> > looks "fine" on a low DPI monitor, but is completely unreadable on a
> > hidpi screen (at least for my old eyes), so the goal would be to
> > scale it up, not down.
> > 
> > The scale factor of 2 advertised in wl_output would be for Wayland
> > native clients, so they can set their buffer scale to that scale 2,
> > and then be downscaled by the compositor to achieve the expected
> > fractional scale of 1.5 with best rendering on screen.
> 
> Oh yes, I forgot that the X11 apps in this case are always drawing with
> scale=1, so they are expected to be scaled up indeed. I must have been
> thinking about all of X11 land being configured with scale=2.
> 
> > If the compositor was to scale Xwayland surfaces by 2 as well and
> 
> I think you mean the X11 apps are configured to draw in scale=2, right?
> 
> > then downscale to achieve the expected fractional scale of 1.5, it
> > would downscale by a factor of (2÷1.5) = 1.33, in which case it would
> > simply advertise a logical size of 5120×2880 (i.e [3840 × 2 ÷
> > 1.5]×[2160 × 2 ÷ 1.5]) - That's the whole idea behind the logical
> > size, whatever the compositor does eventually, Xwayland gets it right
> > as long as the compositor advertise the expected size once the
> > scaling factor -whatever it is- is applied, if that makes any sense.
> > 
> > At least it's how I understood it from the description from:
> > 
> > https://mail.gnome.org/archives/gnome-shell-list/2017-June/msg00000.html
> > 
> > But I might be wrong in my understanding :)
> 
> Yes, now we are onto something. This feels good, considering outputs in
> isolation. I didn't actually go through the numbers yet to make sure the
> example is right, though.

Telling Xwayland the logical output is twice as large as the logical
output actually is, then assuming surface are scale=2 would, I think,
make HiDPI aware clients draw properly, but it'd also make things like
xterm and gitk become completely unusable always right?

> 
> > > I am so confused, because it feels like you are inventing scaling
> > > factors here and there to get the numbers you want, without any
> > > design on what the coordinate systems actually are. OTOH, I always
> > > think in coordinate systems and how to map from one to another, how
> > > you move on the 2D plane and how that motion translates to the
> > > other one, especially when crossing from a low-dpi output to a
> > > high-dpi output. I would love to see the coordinate systems drawn
> > > out, and the mappings between them written down, parametrized with
> > > what we intend to have in protocol and in compositor scaling
> > > modes.  
> > 
> > I read this as a question about fractional scaling design, not how
> > Xwayland would advertise output size.
> 
> They are all related. Xwayland itself together with all X11 clients use
> a coordinate system, and everything needs to be consistent: RandR,
> window placement, buffers, input.
> 
> > > I also cherish the idea of a chain of coordinate transformations.
> > > Start with coordinate system 1, apply transformation A, end up in
> > > coordinate system 2, apply transformation B, end up in coordinate
> > > system 3. All transformations are invertible. If there is more than
> > > one chain of transformations between two coordinate systems, the
> > > total transformations must be identical. The less exceptions on
> > > when each coordinate system applies, the better. This might be
> > > obvious, but it must also hold when traveling in a coordinate
> > > system and crossing window and output boundaries, and that is where
> > > I think we will hit problems. The transformations do not apply only
> > > to points, they must also apply to vectors (motion).  
> > 
> > That's up to the compositor, not Xwayland, innit? Xwayland is just a
> > Wayland client and has no control on that.
> 
> No, I believe Xwayland converts input coordinates from per-wl_surface
> to global X11 coordinates, and then does everything in the global X11
> coordinate system which is all that X11 apps ever know about.
> 
> ISTR there have been Xwayland patches going back and forth on that
> design, so I'm not sure what the current state actually is. It
> definitely is tricky code in Xwayland.

IIRC it'll just take the x/y in surface local coordinates and append the
window position to get the "X" coordinate.

> 
> 
> > > > Yes, but that's for "regular" Wayland clients, Xwayland surface
> > > > aren't scaled in mutter even though Xwayland doesn't use
> > > > wl_surface.set_buffer_scale (mutter doesn't support
> > > > set_buffer_transform, either)  
> > > 
> > > Right, I think that's a mistake. Or at least it makes me not
> > > understand at all how Mutter can work in a mixed-dpi setup.  
> > 
> > You mean mutter or Xwayland with mutter? Either way, it's a comment
> > for mutter devs.
> 
> Mutter with Xwayland, yes, and more precisely the partial HiDPI support
> which is fundamental in getting X11 apps show up readable and not blown
> up.
> 
> We actually can make even plain old X11 apps scale up properly on HiDPI
> outputs, if the Wayland compositor's design allows it. And it sounds
> like the new coordinate systems work in Mutter is going to allow that,
> while the old way of never scale any X11 window was as bad as native
> X11 on HiDPI.
> 
> So my criticism here is against caring at all about the old way of
> never scaling any X11 window in the Wayland compositor. Maybe I have
> misunderstood and you don't care about it?

Caring about the fractional scaling use case and implementing it this
way (communicating the output layout in the global compositor coordinate
space) has the nice side effect of also making it possible to implement
the "old" mutter way of dealing with X11 clients, i.e. regress as little
as possible by not ever scaling X11 client windows.


Jonas

> 
> 
> Thanks,
> pq