[RFC] Visual Class for On-Screen HDR Drawables

[Alex Goins]

Pinging in case this got buried.

Thanks,
Alex

On Wed, 1 Mar 2017, Alex Goins wrote:

> Thanks Adam. Inline -
> 
> On Fri, 10 Feb 2017, Adam Jackson wrote:
> 
> > On Thu, 2017-02-09 at 19:06 -0800, Alex Goins wrote:
> > > Any input?
> > 
> > Inline below...
> > 
> > > > Would it be a major problem for automatic redirection to destroy HDR?
> > > > Compositors could disable automatic redirection for DeepColor windows, and
> > > > redirect themselves with preserved HDR.
> > 
> > It's not a problem in the sense that the actual HDR window content
> > would still exist somewhere; we just need to be able to do _something_
> > when an HDR window ends up in Automatic redirection. We could skip
> > painting it to the parent, we could clamp to some range...
> 
> Right, as long as it's acceptable for it to be done on a best-effort basis I
> think that makes sense, and if we can get away with skipping painting it
> altogether in the worst case that gives us some flexibility to add new formats
> later without breaking compatibility with older versions.
> 
> > > > One of the reasons why we initially went with the idea of a visual opaque to
> > > > core/Render operations is that there are other HDR encodings that have different
> > > > transfer functions from the HDR format to sRGB. It sounds like the current train
> > > > of thought with clamping to the sRGB range assumes that we are operating only on
> > > > the linear FP16 encoding of scRGB, which is highly limiting. scRGB is useful if
> > > > we are trying to composite HDR and SDR content together, but there are other
> > > > formats as well. Even the scRGB color space has support for a 12-bit nonlinear
> > > > encoding in addition to the 16-bit linear one. Android and Windows both have
> > > > support for multiple HDR formats.
> > 
> > Sure. Picking sRGB here is just a plausible default. What we're
> > trying to cover by exposing HDR content through existing visual classes
> > and imaging ops is just "do something reasonable for compatibility with
> > non-HDR-aware apps". If there's some other transformation to unorm8
> > that would produce better visual results along those paths then great,
> > we just need to figure out how to expose that to a) application control
> > b) compositor visibility. There are options there, it could vary by
> > visual (although I think that's not how GLX visuals want to work), it
> > could be some new bit of per-window state that we add to Fixes or XC-
> > MISC or a new extension, it could be a new NETWM property...
> 
> I don't have much of an opinion on whether we should define a whole new
> extension or extend an existing one, although from reading the description of
> the XFIXES extension it doesn't seem like the right scope, in that we aren't
> trying to eliminate a problem caused by a workaround. I hadn't considered NETWM,
> that could make sense to put pressure on window managers to composite HDR
> content correctly rather than relying on automatic redirection, even if most of
> the info is queryable through an extension.
> 
> More on this topic below.
> 
> > > > One concern that came up internally is handling pixmaps, which specify a depth
> > > > but not a Visual. I looked a bit into Mutter and Compton, and they seem to deal
> > > > with this by querying visual properties from the root window, and assuming there
> > > > aren't any pixmap-specific parameters that would affect the texture target.
> > > > However, since the Pixmap is queried from the Window, they should be able to get
> > > > the parameters from the Visual of the Window backing the Pixmap.
> > 
> > I feel comfortable saying these compositors are wrong and should be
> > using glXQueryDrawable to figure out the fbconfig for the window-
> > pixmap; though, having said that, I'm not sure Xorg's glx would return
> > something sensible.
> > 
> > But again, I think the colorspace is more properly a property of the
> > drawable than of the fbconfig. The EGL HDR specs seem to be written in
> > terms of per-drawable state rather than new configs. So we might need
> > to add a new attribute for QueryDrawable; that's fine, the set of
> > people involved in new GLX extension specs are all already on this
> > list, if not this thread.
> 
> FYI - it's only the texture_from_pixmap texture target attributes that they base
> off of the root window, e.g. GLX_TEXTURE_2D_EXT, GLX_TEXTURE_RECTANGLE_EXT,
> GLX_TEXTURE_FORMAT_RBGA_EXT. They get the fbconfig from the X pixmap's depth,
> along with a few other static attributes, which is still a problem for us since
> with HDR we'll need more info than just depth. The texture target derivation is
> still wrong behavior, however, as it should be done using FBConfig attributes.
> If they want to avoid looking up FBconfig attributes every time a window is
> mapped, a better implementation suggested internally would be to cache FBconfig
> capabilities in a hash map or something based on the visual ID, due to the
> variability of texture format attributes between different visuals/fbconfigs.
> 
> Thanks, after going back over the EGL HDR specs I see what you're getting at.
> With GLX we don't really address colorspace with visuals or fbconfigs -- they
> just address the storage. EGL, being the first to address colorspace explicitly,
> sets the precedent that colorspace is a drawable property. That way, we don't
> need a different fbconfig for every interpretation of FP16 component storage. I
> agree with that direction, especially considering EGL's precedent.
> 
> The glXQueryDrawable() method of finding the fbconfig could be a problem with
> our driver, and possibly others per the spec. glXCreateWindow() doesn't
> guarantee that the GLX window and X window are the same, despite being
> associated, e.g.:
> 
>     glXWin = glXCreateWindow(xWin);
>     assert(glXWin == xWin);
>     glXQueryDrawable(xWin);
> 
> will work in Mesa, but with our driver the assert will fail, and
> glXQueryDrawable() should too; it certainly can per the spec.
> 
> If glXWin != xWin, the composite manager can't rely on glXQueryDrawable(xWin)
> when it discovers a redirected window xWin. So, if we're going to use
> glXQueryDrawable(), we'd want a more reliable way to ascertain the GLX window
> XID from the native X window.
> 
> > > > The only problematic case would be an app that has a Pixmap of unknown
> > > > origin/format and wants to texture from it. For GLX, we might have difficulties
> > > > finding a correct fbconfig to create a GLXPixmap to use with glXBindTexImage.
> > 
> > If this is drawable state not config state, then this difficulty
> > disappears, right? You look it up from the window whose pixmap you've
> > bound. Though you do need to get notified if it changes, so we may also
> > need a new event.
> 
> Beyond the above caveat, yeah it should solve the problem for GLX. However, in
> EGL, you can texture directly from a native pixmap without creating an EGL
> surface for it, so we'd need a way to find the pixmap format data at EGLImage
> creation time. There's been discussion about passing it via eglCreateImage()
> attributes.
> 
> If we make format data a property of the native X window, compositors should
> have no issues querying it from the redirected window, and we could piggyback
> off of it to do the transformation to unorm8 based on the specified format if we
> have a known transformation for it.
> 
> If it's also a property of native X pixmaps such that composite window-pixmaps
> share the attributes of their corresponding redirected windows, it would be
> ideal if drivers had a way of intercepting that data before any application can
> attempt to bind those pixmaps as textures.  That way, EGL binding a native X
> pixmap would work like binding any EGL image target, in that format information
> is implied by the target.
> 
> > - ajax
> > 
> 
> It makes sense for colorspace info to be stored as a property of the drawable,
> i.e. scRGB. EXT_gl_colorspace_bt2020 also sets precedent for storing encoding
> along with the colorspace, e.g. BT2020 linear, BT2020 pq.
> 
> Encoding can impact storage format. With scRGB, there are two encodings: 16 bpc
> linear, and 12 bpc nonlinear. To confirm, we're not planning on exposing any of
> that information via X visuals, only using X visuals to interpret the resulting
> pixel data after converting to unorm8, correct?
> 
> Automatic redirection would have to imply the properties of the underlying
> storage from the colorspace/encoding, which may not be ideal. Maybe we could
> expose extended X visual properties as well, for reading the raw HDR pixel data
> before transforming it based on the colorspace/encoding information garnered
> from the drawable properties.
> 
> Thanks,
> Alex