[Mesa-dev] [PATCH] st/vdpau: avoid sending buffers with 10bit channels to X

[Michel Dänzer]

On 19/01/17 02:23 AM, Nayan Deshmukh wrote:
> On Wed, Jan 18, 2017 at 9:21 PM, Michel Dänzer <michel at daenzer.net> wrote:
>> On 19/01/17 12:27 AM, Nayan Deshmukh wrote:
>>> PresentPixmap only works if the pixmap depth matches the window
>>> depth, otherwise it returns a BadMatch protocol error.
>>>
>>> Signed-off-by: Nayan Deshmukh <nayan26deshmukh at gmail.com>
>>> ---
>>>  src/gallium/state_trackers/vdpau/output.c        | 6 ++++++
>>>  src/gallium/state_trackers/vdpau/presentation.c  | 6 +++---
>>>  src/gallium/state_trackers/vdpau/vdpau_private.h | 1 +
>>>  3 files changed, 10 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/src/gallium/state_trackers/vdpau/output.c b/src/gallium/state_trackers/vdpau/output.c
>>> index 98a8011..4466483 100644
>>> --- a/src/gallium/state_trackers/vdpau/output.c
>>> +++ b/src/gallium/state_trackers/vdpau/output.c
>>> @@ -75,6 +75,12 @@ vlVdpOutputSurfaceCreate(VdpDevice device,
>>>
>>>     memset(&res_tmpl, 0, sizeof(res_tmpl));
>>>
>>> +   if (rgba_format == VDP_RGBA_FORMAT_B10G10R10A2 ||
>>> +       rgba_format ==  VDP_RGBA_FORMAT_R10G10B10A2)
>>> +      vlsurface->send_to_X = false;
>>> +   else
>>> +      vlsurface->send_to_X = true;
>>
>> This isn't sufficient: The window depth could be != 24 (in particular 16
>> or 15, or 30). Even if the depths match, the result won't look correctly
> In case the depth is not 24 then vl_winsys_dri3.c will return null and
> it will follow the older code path.

Okay, then it could be as easy as changing the logic above to

    vlsurface->send_to_X = rgba_format == VDP_RGBA_FORMAT_B8G8R8A8;

(this is the format mpv uses, which seems to work correctly :)


-- 
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer