Mesa (main): frontends/va: WA for ffmpeg 10bit encoding crash

[GitLab Mirror]

Module: Mesa
Branch: main
Commit: a585d95803ca38d42b893603a85bf6442a54838a
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=a585d95803ca38d42b893603a85bf6442a54838a

Author: Ruijing Dong <ruijing.dong at amd.com>
Date:   Tue Jul 12 14:40:19 2022 -0400

frontends/va: WA for ffmpeg 10bit encoding crash

When doing 10bit encoding in ffmpeg it uses
VaDeriveImage, and that could result in missing
mapping the chroma buffer of the input frame.

This WA to disallow ffmpeg using VaDeriveImage
function, so that VaCreateImage and VaPutImage can
be used and WA the chroma buffer mapping issue.

Reviewed-by: Leo Liu <leo.liu at amd.com>
Signed-off-by: Ruijing Dong <ruijing.dong at amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17472>

---

 src/gallium/frontends/va/image.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/src/gallium/frontends/va/image.c b/src/gallium/frontends/va/image.c
index ff0ece6237c..19d0f69121a 100644
--- a/src/gallium/frontends/va/image.c
+++ b/src/gallium/frontends/va/image.c
@@ -223,6 +223,10 @@ vlVaDeriveImage(VADriverContextP ctx, VASurfaceID surface, VAImage *image)
          "hevcencode"
    };
 
+   const char *derive_progressive_disallowlist[] = {
+         "ffmpeg"
+   };
+
    if (!ctx)
       return VA_STATUS_ERROR_INVALID_CONTEXT;
 
@@ -251,6 +255,10 @@ vlVaDeriveImage(VADriverContextP ctx, VASurfaceID surface, VAImage *image)
                                    PIPE_VIDEO_ENTRYPOINT_BITSTREAM,
                                    PIPE_VIDEO_CAP_SUPPORTS_PROGRESSIVE))
          return VA_STATUS_ERROR_OPERATION_FAILED;
+   } else {
+      for (i = 0; i < ARRAY_SIZE(derive_progressive_disallowlist); i++)
+         if ((strcmp(derive_progressive_disallowlist[i], proc) == 0))
+            return VA_STATUS_ERROR_OPERATION_FAILED;
    }
 
    surfaces = surf->buffer->get_surfaces(surf->buffer);