Mesa (staging/20.0): st/vdpau: Only call is_video_format_supported hook if needed

[GitLab Mirror]

Module: Mesa
Branch: staging/20.0
Commit: 5376a799628259b51b432a6a746ff68f13b07ed1
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=5376a799628259b51b432a6a746ff68f13b07ed1

Author: Michel Dänzer <mdaenzer at redhat.com>
Date:   Tue Feb 18 12:12:01 2020 +0100

st/vdpau: Only call is_video_format_supported hook if needed

Namely only if *is_supported is true, otherwise the hook result can't
affect it.

Avoids

../src/gallium/state_trackers/vdpau/vdpau_private.h:138: FormatYCBCRToPipe: Assertion `0' failed.

with assertions enabled.

Fixes: 5d5b414a7b84 "st/vdpau: fix chroma_format handling in
                     VideoSurfaceQueryGetPutBitsYCbCrCapabilities"

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer at amd.com>
Reviewed-by: Christian König <christian.koenig at amd.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3848>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3848>
(cherry picked from commit 7e6010106fb3c4eb5436de869183e857243c1006)

---

 .pick_status.json                        |  2 +-
 src/gallium/state_trackers/vdpau/query.c | 14 +++++++-------
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/.pick_status.json b/.pick_status.json
index 94d53f2384d..36c0c588f09 100644
--- a/.pick_status.json
+++ b/.pick_status.json
@@ -256,7 +256,7 @@
         "description": "st/vdpau: Only call is_video_format_supported hook if needed",
         "nominated": true,
         "nomination_type": 1,
-        "resolution": 0,
+        "resolution": 1,
         "master_sha": null,
         "because_sha": "5d5b414a7b840a4a90050041fabd88a9af6dca43"
     },
diff --git a/src/gallium/state_trackers/vdpau/query.c b/src/gallium/state_trackers/vdpau/query.c
index 5f644bbeea0..eca1b3031c1 100644
--- a/src/gallium/state_trackers/vdpau/query.c
+++ b/src/gallium/state_trackers/vdpau/query.c
@@ -154,13 +154,13 @@ vlVdpVideoSurfaceQueryGetPutBitsYCbCrCapabilities(VdpDevice device, VdpChromaTyp
       break;
    }
 
-   *is_supported &= pscreen->is_video_format_supported
-   (
-      pscreen,
-      FormatYCBCRToPipe(bits_ycbcr_format),
-      PIPE_VIDEO_PROFILE_UNKNOWN,
-      PIPE_VIDEO_ENTRYPOINT_BITSTREAM
-   );
+   if (*is_supported &&
+       !pscreen->is_video_format_supported(pscreen,
+                                           FormatYCBCRToPipe(bits_ycbcr_format),
+                                           PIPE_VIDEO_PROFILE_UNKNOWN,
+                                           PIPE_VIDEO_ENTRYPOINT_BITSTREAM)) {
+      *is_supported = false;
+   }
    mtx_unlock(&dev->mutex);
 
    return VDP_STATUS_OK;