[PATCH] Add aspect-ratio support in wayland layer.

[Pekka Paalanen]

On Wed, 20 Sep 2017 11:38:15 +0530
"Sharma, Shashank" <shashank.sharma at intel.com> wrote:

> Hello Pekka,
> 
> Sorry for the delay in response, I was OOO. I had a quick syncup with 
> Ankit, where he told me about the discussions you guys had, over this 
> topic.
> Please find my comments inline.
> 
> On 9/12/2017 2:49 PM, Pekka Paalanen wrote:

> > The policy and configuration to choose the appropriate video mode lies
> > with each compositor. In Weston's case those are weston.ini for
> > configuration and the DRM-backend has the policy. Weston's policy is
> > simple and probably inadequate as it is. The way to fix that is to
> > patch Weston.
> >
> > How to pick the right mode is a question that has no answer without a
> > specific use case and the environment. I envision moving the policy
> > from the DRM-backend to the users of libweston (into e.g. Weston) in
> > the future.
> >
> > For example for a desktop, I struggle to imagine why anyone would want
> > to pick a video mode with non-square pixels. All generic software
> > assumes pixels are square. I would assume that also display panels are
> > built with square pixels, no?
> >
> > For TV and such uses, I know nothing at all except for the feeling that
> > the video standards are still crippled by the designs from the 1960s.
> >
> > If a compositor is being used for a special, non-desktop purpose, I
> > would also imagine it encodes its own configuration and policy for
> > choosing video modes. The choice requires case specific knowledge.
> > Clients could also be involved via domain specific Wayland protocol
> > extensions.  

Hi Shashank,

I added some empty lines below just for readability. My reply is at the
end.

> These are some very valid points, and I agree to most of them. But 
> probably we should discus about little bit about the theory of aspect 
> ratio here.

> There are three variants of aspect ratios which affect the 
> screen display
> 1. Display aspect ratio (DAR): Its the ratio of physical 
> dimension of TV/monitor (width mm / height mm)
> 2. Storage aspect ratio 
> (SAR): Its the ratio of the pixels of the picture captured at the 
> source. For example, a 1920x1080 image, SAR would be 1920:1080 = 16:9
> 3. Pixel aspect ratio (PAR): The aspect ratio of the pixels being displayed 
> on the monitor.

> So an image of resolution 1920x1080 (SAR 16:9) when 
> being displayed on a TV of (DAR 16:9) will have square pixels of (PAR 
> 1:1). So from the definition, we can deduce that PAR = DAR/SAR

> While 
> sending the AVI info-frames from source to sink, the source set aspect 
> ratio field, and the monitor prepares itself accordingly. In fact, there 
> is a HDMI compliance certification test (7-27) which checks if the 
> selected aspect ratio is being reflected in AVI IF correctly. Now, there 
> are many widescreen TVs and gaming monitors available in market, where a 
> selected gaming app / media player / dvd player / set-top box would want 
> to show non-square pixels for:
> - widescreen effect / theater effect.
> - gaming experience.
> - VR/AR and few other modern display/gaming 
> techniques.

> Also, HDMI 2.0 standard / CEA-861-F have introduced new 
> aspect ratios like 64:27 and 256:135 so that modern games / media 
> playback can utilize these monitors / displays well.

> So considering 
> these facts, I was thinking that it would be under utilizing the monitor 
> capabilities and it may restrict the experience if we don't consider the 
> desired widescreen intention of the compositor / app. But at the same 
> time, I might have pulled the stuff too early, and right now when we are 
> in the stabilization stage, I guess it would be good to add the feature 
> at the Weston layer, and defer this activity for later once we have 
> proper infrastructure in place. So I liked the suggestions where we add 
> this later as an extension for Set Top Box/Media/IVI usages. I have 
> asked Ankit to re-tune the patch series as per your suggestions, and 
> restrict aspect ratio feature into drm-compositor only. He will probably 
> publish it today. Please have a look  into that.
> Regards Shashank

I very much appreciate the explanation! Thank you.

However, I am still left wondering about the fundamental reasons on why
anyone would want to see an image streched away from its SAR. Unless,
the content originally had PAR not 1:1, or the display device has PAR
not 1:1. Why anyone would have PAR not 1:1 is beyond me, the only
reason I can imagine is for historical reasons (like why do interlaced
modes still exist in the time of digital imaging) stemming from early
analogue technology.

I am happy to hear that you are satisfied with the suggestion to use
domain specific protocol extensions for these special use cases.

As for the desktop, or all existing Wayland usages today really, we are
running with the assumption that PAR is 1:1, but we do have a way for
clients to express content in different PAR: wp_viewporter extension.
It allows the client to define the source and destination rectangle
sizes which can be used to denote an arbitary PAR in the original video
content.

Usually a compositor would ensure the that content gets stretched
accordingly to a display device with a PAR 1:1. If the compositor has
been built to handle arbitrary display device PAR, then when the device
and content PAR match, there would be no scaling applied by the
compositor.

Furthermore, a compositor could already make the decision to switch
video modes on the fly as needed. Say, you have a fullscreen window,
whose content is given at a certain PAR. Assume the display device also
exposes a video mode with a matching PAR. The compositor could
temporarily switch to this video mode. Unfortunately libweston's
support for that is quite poor at the moment.

What we are currently lacking here is a way for the client to request a
specific frame rate. There are probably also other details a protocol
extension aimed at video players would like to carry, so that the
compositor can make the optimal decision based on the content type,
user preferences, and the global display system state.

I feel that some video specific protocol features have already leaked
into zwp_linux_dmabuf extension, notably the interlaced and
bottom_first flags.

---

One thing I have recently wondered is, if a compositor chooses a video
mode whose SAR does not match the EDID-encoded DAR, does it mean that
we get PAR not 1:1 or that the physical image area is adjusted to
produce PAR 1:1 (letter/pillar boxing by the device).

I believe we have a way to communicate PAR to clients already:
wl_output has both physical dimensions and the current resolution from
which clients can compute SAR and DAR. But the thing at least Weston
does not do is adjust the physical dimensions according to the other
factors. It does not know about display device PAR. Yet.


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20170925/f71d19aa/attachment.sig>