[Mesa-dev] 10-bit fbconfigs break most video players using VAAPI+GLX

[Adam Jackson]

On Wed, 2019-02-06 at 10:33 -0500, Adam Jackson wrote:
> On Tue, 2019-02-05 at 17:30 -0500, Marek Olšák wrote:
> 
> > If we expose 10-bit or 16-bit formats, a lot of software will be
> > broken. Any ideas how to get out of this rabbit hole?
> 
> Use GLX_SGIX_visual_select_group to hide the depth-30 formats after the
> depth-24 ones. We're already doing this for similar reasons to mask
> Composite's synthetic argb8888 visual on rgb565 displays. Technically
> there's nothing preventing you from doing this purely client-side, but
> properly this is a server-side change (and an easy one).

... although, this may just move the breakage around. I assume the
failure mode here is that the selected fbconfig doesn't match the
visual of the destination window? If so, then preferring the 8888
fbconfigs will still mismatch if the destination window really is
2101010.

(goes and actually reads libva)

So, some of this problem goes away by using GLX_EXT_no_config_context
at context creation time, if available. That at least gives you a
context that can bind to arbitrary drawables. I'm not entirely clear on
how you inform libva which window you expect to draw to, but if you had
that at CreateContext time you could also just specify the
GLX_VISUAL_ID you wanted.

But the other place it uses glXChooseFBConfig is when creating the
offscreen GLXPixmap, and honestly I don't understand why libva bothers
to do that at all. What possible benefit is derived from using
GLX_EXT_texture_from_pixmap for the rendering surface? Is someone
seriously trying to render frames from one X client and display from
another, and if so, how are they getting the GLXPixmap's XID out of
libva? Why make this a server-side resource at all?

If there are other video players / engines that need fixing, I say,
let's fix them. I'm not sure any amount of client-side workaround can
ever be satisfactory.

- ajax