[Mesa-dev] [PATCH 3/4] st/va: properly set max number of reference frames

[Julien Isorce]

It fixes asserts like assert(templ->max_references <= 2) in
nvc0_video.c::nvc0_create_decodier
This patch also post-update the correct value for the number
of reference frames in the h264 case, see below.

In VA-API the max num ref is retrieved later in handlePictureParameterBuffer.
Unfortunately by this time the decoder has been already created
in vaCreateContextwhich which one does not have any max_references
param compared to VDPAU api.

In vdpau-driver they delay the decoder creation to endPicture.
This is not practicable with gallium. But that's ok our buffer
will be bigger but at least they will have enough space and
st/va will still write correct value for the number of reference
frames in the hw buffer.

Signed-off-by: Julien Isorce <j.isorce at samsung.com>
---
 src/gallium/state_trackers/va/context.c | 10 +++++++---
 src/gallium/state_trackers/va/picture.c |  1 +
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/src/gallium/state_trackers/va/context.c b/src/gallium/state_trackers/va/context.c
index ec9e048..b3dd293 100644
--- a/src/gallium/state_trackers/va/context.c
+++ b/src/gallium/state_trackers/va/context.c
@@ -248,10 +248,14 @@ vlVaCreateContext(VADriverContextP ctx, VAConfigID config_id, int picture_width,
       templat.max_references = num_render_targets;
       templat.expect_chunked_decode = true;
 
+      /* XXX HEVC ? */
       if (u_reduce_video_profile(templat.profile) ==
-        PIPE_VIDEO_FORMAT_MPEG4_AVC)
-        templat.level = u_get_h264_level(templat.width, templat.height,
-                             &templat.max_references);
+         PIPE_VIDEO_FORMAT_MPEG4_AVC) {
+         templat.level = u_get_h264_level(templat.width, templat.height,
+            &templat.max_references);
+      } else {
+         templat.max_references = 2;
+      }
 
       context->decoder = drv->pipe->create_video_codec(drv->pipe, &templat);
       if (!context->decoder) {
diff --git a/src/gallium/state_trackers/va/picture.c b/src/gallium/state_trackers/va/picture.c
index 5e7841a..9d4d1a8 100644
--- a/src/gallium/state_trackers/va/picture.c
+++ b/src/gallium/state_trackers/va/picture.c
@@ -146,6 +146,7 @@ handlePictureParameterBuffer(vlVaDriver *drv, vlVaContext *context, vlVaBuffer *
       /*bit_depth_luma_minus8*/
       /*bit_depth_chroma_minus8*/
       context->desc.h264.num_ref_frames = h264->num_ref_frames;
+      context->decoder->max_references = MIN2(context->desc.h264.num_ref_frames, 16);
       /*chroma_format_idc*/
       /*residual_colour_transform_flag*/
       /*gaps_in_frame_num_value_allowed_flag*/
-- 
1.9.1