wl_output ambiguity, xdg_shell fullscreen refresh rate

[Pekka Paalanen]

On Mon, 15 May 2017 17:31:57 +0200
Philipp Kerling <pkerling at casix.org> wrote:

> Thanks for your answers! See reply below.
> 
> 2017-05-15 (月) の 11:18 +0300 に Pekka Paalanen さんは書きました:
> > On Sun, 14 May 2017 14:43:44 +0200
> > Philipp Kerling <pkerling at casix.org> wrote:  
> > > To start off:
> > >  * Kodi should offer the user the opportunity to select inside the
> > >    application on which monitor he/she wants to have Kodi displayed
> > > on
> > >    when in full-screen mode. Now I see that I can do this with
> > > either
> > >    wl_shell_surface::set_fullscreen and
> > > zxdg_toplevel::set_fullscreen,
> > >    which expect a wl_output to display the window on. But I am not
> > > sure
> > >    how I should match the wl_output instance I get from Wayland
> > > with
> > >    the monitor that the user selected. As far as I can see, I can
> > > only
> > >    identify the monitor by way of "make" and "model". This seems
> > >    reasonable enough at first glance, but I think that in multi-
> > > monitor 
> > >    configurations it is not so uncommon to buy the same monitor
> > > model
> > >    multiple times - I personally have done this. Naturally, the
> > > "make"
> > >    and "model" of both monitors are identical, so I cannot discern
> > >    which is which by looking at the wl_output metadata. Am I
> > > missing
> > >    something here?  
> > 
> > That is all true, you are not missing anything.
> > 
> > It is a missing piece of protocol - I believe no-one has had both the
> > need and the time to work on getting anything better yet.
> >   
> > > Why isn't e.g. the connector the output is connected
> > >    to exported?  
> > 
> > Because connector names are specific to DRM while Wayland (core
> > protocol at least) is not.  
> I'm not sure I agree. See further below.
> 
> > Also, would you not want to identify the
> > monitor rather than which connector it happens to be plugged in?   
> I think that it's not quite decided yet what we want the user
> experience to be on the Kodi side, i.e. do we want to bind output to
> the physical monitor (however it is identified) or the GPU connector?
> You can probably find use cases for both, so it's not an easy decision
> to make. With X11 it was limited to the latter, so my first approach
> was to mimic that. Ideally, both would be possible with Wayland and the
> application can decide what is best for its specific use case. I
> realize that this might be too much to ask for all at once, so I'll
> talk to my mentor to decide which case we want to focus on for Kodi.
> There are also two different points involved in this discussion:
>    I. How the *user* identifies which output/monitor the fullscreen
>       display should be on (e.g. with make+model+connector name, or by
>       compositor-specific numbering).
>   II. How *Kodi* or any other application then stores this, re-identifies
>       this output/monitor and maps it to a wl_output on application start
>       and hotplug. Possiblities include saving information pertaining to
>       the output, the connector, the physical device, or any combination.

Hi,

very much agreed.

> > Would
> > a user even know which connector name matches to which monitor?  
> Yes, I also thought about this. Tech-savvy users might actually be able
> to identify the connector name, but you are absolutely right that
> usually users should not be expected to. In fact, having make and model
> available actually is an improvement in that regard (on X11, Kodi
> *only* displays connector since that is what's available via RANDR).
> So: How do you suggest to identify a particular monitor then? As I
> said, make and model are not sufficient - but serial number will not
> help the user in identification. If there are multiple monitors of the
> same model, I do not think that they (i.e. the physical devices) can be
> reasonably discerned in a way that is transparent to the user. That's
> why I suggested the connector name.

One idea coming to mind is that in the DE's (compositor's) display
configuration dialog, the user would be allowed to identify monitors by
visual clues, and would be given the opportunity to give the monitors
(or connectors!) custom names. An application could then show all the
information plus the custom names so that the user can pick the one he
wants.

> > To expose such a name in the protocol, we should define the space or
> > meaning of the names. Human-readable? That's not the connector name,
> > IMHO. Machine-parseable? Needs a very strict definition to work.
> > Both?  
> RANDR output names (which are somewhat close to what I refer to as
> connector names - it's a matter of debate whether output or connector
> name would be more fitting for the Kodi use case) are specifically
> meant to be "presented to the user", but I can see your point. I think
> that machine-parseable is doable, also considering that RANDR did have
> a "ConnectorType" property which is probably part of what we'd want
> here. 

I think you are right and we have to cater for two very different
cases: how a user would identify a connector or a monitor, and how a
program would recognize a previously chosen connector or monitor. It's
also possible one would need to let the user decide if one is matching
the connector or the monitor.

When thinking about naming schemes, it easy to forget about multiple
DRM display devices. I don't think DRM connector names are unique
globally, they are unique for a single DRM device. Any global
naming needs to somehow identify the DRM device too. That's
something the compositor needs to solve, but it has some consequences
on how the protocol specification should be written.

DRM drivers can be initialized in random order, so one cannot rely on
the ordering.

> > A simple numbering with a way in the compositor to identify the
> > monitors? Maybe.  
> What exactly do you mean by "a way in the compositor to identify the
> monitors"? Like you can tell the compositor to display the output names
> (briefly) on the outputs so the user knows which is which?

Yes.

> If so, couldn't this also be done by the application at least with more
> or less arbitrarily assigned numbers by mapping a surface to each known
> output that displays said number?

Not conveniently, and not in a way guaranteed to work. As I wrote in my
other email, Wayland clients ask to do things, but the compositor may
decide otherwise. E.g. it is not guaranteed a new window will be placed
top-most to be visible. Clients also cannot control the positioning
much.

> I'm beginning to think that something along those lines might be one of
> the more user-friendly options of solving I. above.

Yes, but the visual cues for identifying the monitors really must be
in the compositor. Also, once it has been saved in the compositor, e.g.
by assigning custom names, there is no need to ever do it again until
monitors change. IOW, I believe identification is compositor
configuration, not something every app should do on its own.

> > > This would allow way better matching and also give the
> > >    user a chance to select the right monitor without having to try
> > > all
> > >    options. Serial number (or anything similar - not sure if the
> > > EDID
> > >    has this though) could also be OK.  
> > 
> > EDID may have a serial number, yes.
> > 
> > Physical monitor devices are not the only things though. An output
> > could be just a window in another window system. An output could be
> > at
> > the end of a streaming network connection. Etc. So aiming for a
> > generic
> > solution here would have a really wide scope.  
> Also answering the first question above: I don't see how "make" and
> "model" (and the mode list for that matter) in wl_output fit with this
> statement. They are also limited to traditional physical devices
> (monitors), yet are in the core protocol. I don't suppose that it makes
> sense to classify an output in another window system with a
> manufacturer and model. That said, I also don't see why "connector" (or
> something similar) would be fundamentally different from what is
> already there.

You're absolutely right. wl_output is one of the oldest designs in
Wayland, and carries quite many things that could have been designed
better in hindsight. Let's not make it any worse than it already is.

Of course, one could put things like "rtp://..." or "Window 0x65" in
connector attribute, just like one can put arbitrary things in make and
model, but could we not do better? Or do we not need to do any better?

I'm honestly unsure.

However, one thing is certain: if we extend wl_output interface, it
will be immediately set in stone forever, because it's in wayland.xml,
and wl_surface references wl_output objects. There will be no turning
back. The only way to deprecate things is to say they have become
meaningless, but they can never be actually removed.

If instead we write another protocol extension, that extension is much
easier to evolve and can be deprecated and even removed and replaced if
necessary. It will also be optional to implement, for better and worse.

> > However, there already is a unique numbering for wl_outputs, even
> > though it's not exactly user-friendly. It is the wl_output "name"
> > advertised by wl_registry, which you use in wl_registry.bind request
> > to
> > create a wl_output instance. This "name" is the same for all clients.  
> Is this name (which is usually the integer number of the protocol in
> enumeration order if I got that right) *guaranteed* to stay the same
> (for a given wl_output)
>  * when I restart the application?

Yes.

>  * when I restart the compositor?

No.

>  * when I restart the computer?

No.

>  * when I connect additional monitors?

Yes, already existing wl_outputs do not change their "name".

>  * when I rearrange monitors in the compositor?

Depends on what the compositor does.

>  * when I start the application with the monitor unplugged and plug it
>    in later (while the app is running)?

No.

If you unplug an re-plug a monitor into the same connector, the "name"
will be different.

"For a given wl_output" is a bit awkward here, because the "name"
defines the wl_output identity. I assume you wanted to refer to a
connector or a monitor instead.

> 
> If not, this would not be useful to Kodi. Whether other clients see the
> same "name" or not is not especially relevant here. The specific use
> case is to save the monitor/output identifier to the configuration file
> and have it persist across both app and device restarts and output
> configuration changes. We would especially like to support the last
> point of the enumeration above, i.e. starting on a fallback
> (unspecified) monitor and then switching fullscreen output to the
> preferred monitor once it is connected by the user.

Right.

What about across different compositors?

If a user runs Kodi on one compositor and configures it, switches to
another compositor, do you still expect the saved configuration to be
valid?

I assume you do, which makes it crucially important to define in the
protocol specification on what names or other information to put in
wl_output attributes. When one needs to be so excruciatingly accurate
in the specification, you pretty much want to restrict the scope to
e.g. DRM display devices only or refer to specific EDID fields.

> > What we would really need is a new protocol extension designed to
> > cater
> > for specific use cases and offering the additional information to
> > help
> > make user-friendly UIs. Maybe compositors should a) number of the
> > outputs, and b) let the user give them arbitrary names to be shown in
> > application UIs?  
> I don't see the real benefit in b), it is IMHO too much effort for the
> average user. I'm also not sure what I would personally label my
> screens as besides "Left screen" and "Right screen", which the
> compositor could as well figure out on its own :-)

It would be an option to customize, in case the default names
manufactured from DRM connector, make, model etc. are not good enough.
Yes, "left screen" and "right screen" are also good default names, but
it still means that the protocol specification has to have a way to
communicate them to clients.

> > But then, do you tie the output identity to the
> > connector or the physical device? Or an intelligent mix?  
> As I said, best is probably if the application has enough information
> to be able to do either.
> 
> > Would it be in scope for your work to design such an extension? It's
> > relatively easy to write a protocol proposal, but getting it reviewed
> > and hopefully ack'd by more than one DE project is often hard and
> > time
> > consuming. And then you'd need to provide an implementation on at
> > least
> > one compositor, unless you find someone else to collaborate with you.  
> If the consensus is that this will, in some form or another, need a
> protocol extension, I'll have to discuss with my mentor whether this
> can be pursued as part of my GSoC work, especially the reference
> implementation part. I think that it is a possibility, but does require
> that there is a realistic expectation that at least mutter or KWin will
> support the extension in the future. Otherwise it could be very hard to
> justify.

I do believe an extension is necessary sooner or later. We can also use
Weston for proving a sample implementation, but that won't help your
users. If you decide to tackle this, I warmly recommend contacting the
DE projects ASAP. Who knows, maybe someone has started on something
already.

> > >    quite a problem. I can see that zxdg_toplevel functionality is
> > >    mostly superior to that of wl_shell_surface, but it has one
> > > omission
> > >    that is crucial for Kodi: the ability to request a specific
> > > refresh
> > >    rate for fullscreen display. This is needed for closely matching
> > > the
> > >    display and video FPS so duplicated and skipped frames are kept
> > > to a
> > >    minimum. Is this an intentional omission and/or is there
> > > anything
> > >    that provides this functionality?  
> > 
> > Hopefully the xdg_shell designers would answer that, but I believe it
> > has been omitted for now to make it easier to declare the first
> > fundamental parts of xdg_shell stable. It is missing a lot, but the
> > foundation must agreed upon first before building more on top.  
> So is there any realistic chance to have such functionality integrated?
> What steps would have to be done?

My personal piece of advice: do not make it a blocker for your own
work. xdg_shell has been in development for years by now and still has
not been declared stable.

I don't think people would disagree much that some way of communicating
video timing related information would be good, but beyond that I can't
tell. It might be best to restrict this out of your project scope.

For example: variable refresh rate monitors exist and the software
infrastructure is slowly building up, I believe. There it is
questionable if one would want windows to declare their optimal refresh
rates, or just submit content with target timestamps for the compositor
to arrange everything. Some years ago when I was working on the
Prensentation timing extension, it also included a feature to
queue window content updates with target timestamps. That feature was
hard to define well, and eventually I abandoned it to get at least the
feedback part stabilized, which is the current Presentation timing
extension.


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20170516/8f387e72/attachment-0001.sig>