[PATCH weston] xdg-shell: Make stable

[Pekka Paalanen]

On Tue, 26 Aug 2014 17:50:47 +0300
Giulio Camuffo <giuliocamuffo at gmail.com> wrote:

> 2014-08-26 17:39 GMT+03:00 Jason Ekstrand <jason at jlekstrand.net>:
> >
> > On Aug 26, 2014 1:01 AM, "Giulio Camuffo" <giuliocamuffo at gmail.com> wrote:
> >>
> >> 2014-08-26 10:24 GMT+03:00 Pekka Paalanen <ppaalanen at gmail.com>:
> >> > On Mon, 25 Aug 2014 21:51:57 -0700
> >> > Jason Ekstrand <jason at jlekstrand.net> wrote:
> >> >

> >> >> On Aug 22, 2014 1:48 PM, "Jasper St. Pierre" <jstpierre at mecheye.net>
> >> >> wrote:
> >> >> >
> >> >> > On Wed, Aug 6, 2014 at 9:39 AM, Pekka Paalanen <ppaalanen at gmail.com>
> >> >> wrote:
> >> >> >>
> >> >> >> On Thu, 17 Jul 2014 17:57:45 -0400
> >> >> >> "Jasper St. Pierre" <jstpierre at mecheye.net> wrote:
> >> >> >>

> >> >> >> >       <entry name="fullscreen" value="2" summary="the surface is
> >> >> fullscreen">
> >> >> >> >         The surface is fullscreen. The window geometry specified
> >> >> >> > in
> >> >> the configure
> >> >> >> >         event must be obeyed by the client.
> >> >> >>
> >> >> >> Really? So, will we rely on wl_viewport for scaling low-res apps to
> >> >> >> fullscreen? No provision for automatic black borders in aspect ratio
> >> >> >> or
> >> >> >> size mismatch, even if the display hardware would be able to
> >> >> >> generate
> >> >> >> those for free while scanning out the client buffer, bypassing
> >> >> >> compositing?
> >> >> >
> >> >> >
> >> >> >> Since we have a big space for these states, I suppose we could do
> >> >> >> those
> >> >> >> mismatch cases in separate and explicit state variants of
> >> >> >> fullscreen,
> >> >> >> could we not?
> >> >> >
> >> >> >
> >> >> > I explicitly removed this feature from the first draft of the patch
> >> >> simply to make my life easier as a compositor writer. We could add
> >> >> additional states for this, or break fullscreen into multiple states:
> >> >> "fullscreen + size_strict" or "fullscreen + size_loose" or something.
> >> >> >
> >> >> > I am not familiar enough with the sixteen different configurations of
> >> >> > the
> >> >> > old fullscreen system to make an informed decision about what most
> >> >> > clients
> >> >> > want. Your help and experience is very appreciated. I'm not keen to
> >> >> > add
> >> >> > back the matrix of configuration options.
> >> >>
> >> >> Yeah, not sure what to do here.  I like the idea of the compositor
> >> >> doing it
> >> >> for the client.  Sure, you could use wl_viewport, subsurfaces, and a
> >> >> couple
> >> >> black surfaces for letterboxing.  However that is going to be far more
> >> >> difficult for the compositor to translate into overlays/planes than
> >> >> just
> >> >> the one surface and some scaling instructions.
> >> >
> >> > I don't think it would be that hard for a compositor to use overlays
> >> > even then. Have one surface with a 1x1 wl_buffer, scaled with
> >> > wl_viewport to fill the screen, and then have another surface on top
> >> > with the video wl_buffers being fed in, scaled with wl_viewport to keep
> >> > aspect ratio. A compositor can easily put the video on an overlay, and
> >> > if the CRTC hardware supports, it might even eliminate the black surface
> >> > and replace it with a background color.
> >>
> >> What is not clear to me is what advantages does the
> >> wl_subsurface+wl_viewport approach has compared to the compositor just
> >> scaling the surface and putting a black surface behind it.
> >> Is it just to remove the hint in the wl_fullscreen request? This seems
> >> a lazy reason to me, implementing that hint in the compositor is not
> >> hard, and in turn it means increasing the complexity of every client
> >> that would ever want to go fullscreen.
> >> There is also one use case for which the wl_subsurface+wl_viewport
> >> cannot work. Having the same surface fullscreen on two differently
> >> sized outputs (think of presentations), like this:
> >> http://im9.eu/picture/phx647
> >> Sure, the compositor can send the configure event with one output's
> >> size and scale the surface to fit the other one, but then what is the
> >> purpose of wl_viewport again here, if the compositor must scale the
> >> surface anyway?
> >
> > Yeah, I think I still have to go with Guilio here.  It really it's probably
> > less work for the compositor to implement fullscreen modes than to implement
> > viewport+subsurface correctly and it's certainly less work for clients.
> >
> >> >
> >> > In my previous reply, I concluded that wl_viewport+wl_subsurface would
> >> > be enough (I suprised myself), and we would not really need yet another
> >> > way from xdg_surface. Obviously, I forgot something, that the IRC
> >> > discussion of last night brought back to my mind.
> >> >
> >> > The wl_shell fullscreening has three different cases for dealing with
> >> > size mismatch between the output and the window:
> >> > - aspect-correct scaling
> >> > - centered, no scaling
> >> > - please, I would really like a mode switch
> >> >
> >> > There is also the fourth, which means the client does not care at all
> >> > what the compositor does. This protocol was designed before
> >> > sub-surfaces or wl_viewport were a thing.
> >> >
> >> > If we do not have that in xdg_shell, but instead rely on
> >> > wl_viewport+wl_subsurface, there are two consequences:
> >> > - scaling vs. centered is no longer a hint, but it is dictated by the
> >> >   client
> >> > - the mode switch case is lost
> >> > (- all desktop compositors are required to implement both wl_scaler
> >> >   and wl_subcompositor)
> >> >
> >> > The first point is likely not an issue, but the second may very well
> >> > be. If xdg_surface requires fullscreen windows to be the exact output
> >> > size, and clients obey, there is no way to trigger the mode switch.
> >> >
> >> > I suspect there are at least gamers out there, who would not like this.
> >> > In fact, they would be pushed back to the way X11 works right now: if
> >> > you want a mode switch, use a helper app to permanently change the
> >> > output resolution (this screws up your whole desktop layout), then
> >> > launch the game, afterwards do a manual switch back.
> >> >
> >> > No, sorry, that would actually be *worse* than the situation on X11
> >> > right now.
> >> >
> >> > Recalling that, I do think we need something to support the mode switch
> >> > case. A crucial part of a video mode for a gamer is the monitor refresh
> >> > rate, which is why wl_shell_surface.set_fullscreen includes the
> >> > framerate parameter.
> >>
> >> I don't think games need the screen to be at a NxM pixels mode,
> >> scaling up the surface would be good and possibly better, since we can
> >> scale better than what LCD screens usually do.
> >> On the other hand, there is the framerate parameter, and games may
> >> care about that... I'm not sure what is the best course of action
> >> here.
> >
> > I'd agree.  Even on X, when you do a "mode switch" it frequently isn't a
> > mode switch at all.  I know that on the nVidia driver, X will do scaling on
> > the graphics card if it's plugged in via a digital connection (DVI, HDMI,
> > DP, etc.)  There really is no "mode switch" involved.  This is precicely
> > because of what Guilio mentioned:  LCD monitors do crappy scaling.

Yes, I know that, and the driver scaling has bitten me when that leads
to *down*scaling, but that is completely irrelevant here. :-)

> > That said, there's a counter example!  Look at LCD projectors.  Unless you
> > have the projector oriented perfectly so that no keystoning is required
> > (BTW: this never happens in practice), the projector will transform the
> > image you give it to make it more rectangular on the projected screen.  In
> > this case, the graphics card stretching it and the projector transforming it
> > will probably look worse than the projector transforming the original image.
> >
> > I think the point here is that the app can't really know whether it wants
> > scaling or a mode switch.  Apps shouldn't need to be dealing with all the
> > details of what outputs are connected and how beyond maybe knowing that the
> > presentation goes on the projector.  Compositors are much better situated to
> > decide when do a mode-switch vs. putting the fullscreen surface in a plane.

Well, as I've tried to explain, "mode switch" is not a real mode
switch, it is just a wish. The compositor chooses the best course of
action, also based on the user's preferences.

Fullscreening is still about going to one wl_output, so you have to
care about outputs. Outputs have a mode list, and apps may or may not
use that list as options to show to the user for fullscreen window
size. The mode list would probably be just a supplement to the app's
own hardcoded list, plus maybe completely freely chosen resolution. The
only thing I see is that it should help the user to pick a good
resolution, if the user really has to go changing it.

> > I think the bigger issue here is framerate.  The reason why games want to
> > run at these odd modes is because your card may not be able to render the
> > game at 1920x1080 at 60fps (to use an extreme example).  So what does the game
> > do?  It asks for a lower resolution and maybe a lower framerate.  If it can
> > only hit 50fps consistently, then using a 50hz vsync looks better than the
> > jitter you would get on a 60hz vsync.  Can we use a heuristic for this?
> > Maybe, maybe not.

Video playback is another thing. Some people really want the refresh
rate to match the video framerate.

> >> > Also remember, that the mode switch is not meant to be mandatory if
> >> > requested by a client. It is only a preference, that when this app is
> >> > active (and top-most?), it would really like the video mode to be
> >> > changed to the nearest compatible with the window. The compositor is
> >> > free to switch between the game's mode and the native desktop mode at
> >> > will. Minimize the game? Sure, switch back to the native desktop video
> >> > mode. Bring the game back - switch back to the game mode. Alt+tab?
> >> > Switch only when something else than the game gets activated, maybe.
> >
> > As I mentioned above, I don't think clients should care whether they get a
> > new mode or not beyond a possible FPS change.  However, I don't know that I
> > like the word "hint" in the fullscreen spec in wl_shell_surface.  The way he
> > spec was written before, the client could ask for centered and get a
> > mode-switch or vice-versa.  What was the intent here?

Clients would not know if they got a new mode in any case.

> > Perhaps we can do everything we want by saying that fullscreen means "fill
> > the screen without changing aspect-ratio" and add a couple of hints requests
> > that the client calls before calling set_fullscreen:
> >
> > - Preferred framerate:  "I don't think I can render any faster than this, so
> > it will look better if you refresh at this rate."  This could, in theory,
> > apply to non-fullscreen windows as well.  Also, it could be a bogus rate
> > such as 24fps (movies) and the compositor could try to refresh at a multiple
> > of 24 or something.
> 
> We should also have a special value for "i don't care", maybe 0.

I think the preferred framerate hint is a very good idea. It is one
part of the essence of this whole thing.

> > - Preferred fullscreen scaling:  "I would like to be as large as possible",
> > "I would like to be pixel-perfect, even if that means smaller and surrounded
> > in black", etc.  TBH, I don't know how many clients would actually like the
> > later one.  Maybe if they have text they don't want getting blurry?
> 
> I also don't see much use for the second (centered) mode. Scaling
> should definitely be the default, imho.
> 
> This is going in the right direction, i think.

This is the question about whether xdg_surface should require the
window to match exactly the configured size. I interpret your replies
as "at most as big as the configured size", allowing to be smaller in
either or both dimensions. Right?

Let's do a summary:
- a request to go fullscreen
- preferred framerate hint, including dont-care option
- allow the fullscreen window to be smaller than configured, defaulting
  to scaling up as big as possible without losing any image content or
  changing aspect ratio

If a client wants any specific kind of scaling / black bars, it can
still do that with wl_viewport and sub-surfaces. This is kind of an
override for special apps with special needs.

A video player might want to use the preferred framerate hint even
while not fullscreen. (Make it an independent xdg_surface request?)

A game may be using window geometry smaller than configured, and set
the framerate hint. The compositor can choose to change video mode
and/or scale up, or not.

The compositor's user preferences might favour integer scaling factors
if the user thinks they are better than fractional. In the game
example, the compositor is free to use an integer factor and fill the
rest with black.

A hardcore gamer using a CRT would set his compositor preferences so,
that the compositor will always switch video mode (perhaps per app) to
the app's fullscreen resolution.

Would that sound good?
Any other use cases to consider?

The two key features would be the framerate hint, and allowing smaller
than configured fullscreen windows.


Thanks,
pq