[Libva] [driver]A fix for vc1 stream of ttmbf parameter

[Zhao, Halley]

Hi:
When gst-vaapi is used for vc1 playback, many stream give corruption image,
after some debug, I found it was caused by incorrect ttmbf parameter:
     in pseudo code , ttmbf is not set when vstransform is zero, however we can rationalize it from context.
I try to fix it in gstreamer vc1 codec parser: https://bugzilla.gnome.org/show_bug.cgi?id=684341

However, Gwenole expects a fix in driver code. So I sent this patch to discuss it in libva list.
Thanks


Patch detail:
= = = = = = = =
>From 08f0d72a63c566e55027bae244774883aa31e25b Mon Sep 17 00:00:00 2001
From: Zhao Halley <halley.zhao at intel.com>
Date: Fri, 21 Sep 2012 10:30:55 +0800
Subject: [PATCH] fix ttmbf/ttfrm when vstransform is 0

---
src/gen6_mfd.c |   14 ++++++++++++--
src/gen7_mfd.c |   13 ++++++++++++-
2 files changed, 24 insertions(+), 3 deletions(-)
mode change 100644 => 100755 src/gen6_mfd.c
mode change 100644 => 100755 src/gen7_mfd.c

diff --git a/src/gen6_mfd.c b/src/gen6_mfd.c
old mode 100644
new mode 100755
index 7f8b2f3..7329d8f
--- a/src/gen6_mfd.c
+++ b/src/gen6_mfd.c
@@ -1656,9 +1656,19 @@ gen6_mfd_vc1_pic_state(VADriverContextP ctx,
     if (picture_type == GEN6_VC1_I_PICTURE || picture_type == GEN6_VC1_BI_PICTURE) /* I picture */
         trans_ac_y = pic_param->transform_fields.bits.transform_ac_codingset_idx2;
-    else
+    else {
         trans_ac_y = pic_param->transform_fields.bits.transform_ac_codingset_idx1;
-
+        /*
+         * 8.3.6.2.1 Transform Type Selection
+         * If variable-sized transform coding is not enabled,
+         * then the 8x8 transform shall be used for all blocks.
+         * it is also MFX_VC1_PIC_STATE requirement.
+         */
+        if (pic_param->transform_fields.bits.variable_sized_transform_flag == 0) {
+            pic_param->transform_fields.bits.mb_level_transform_type_flag   = 1;
+            pic_param->transform_fields.bits.frame_level_transform_type     = 0;
+        }
+    }
     if (picture_type == GEN6_VC1_B_PICTURE) {
         struct gen6_vc1_surface *gen6_vc1_surface = NULL;
diff --git a/src/gen7_mfd.c b/src/gen7_mfd.c
old mode 100644
new mode 100755
index d8581ed..2d75baf
--- a/src/gen7_mfd.c
+++ b/src/gen7_mfd.c
@@ -1633,8 +1633,19 @@ gen7_mfd_vc1_pic_state(VADriverContextP ctx,
     if (picture_type == GEN7_VC1_I_PICTURE || picture_type == GEN7_VC1_BI_PICTURE) /* I picture */
         trans_ac_y = pic_param->transform_fields.bits.transform_ac_codingset_idx2;
-    else
+    else {
         trans_ac_y = pic_param->transform_fields.bits.transform_ac_codingset_idx1;
+        /*
+         * 8.3.6.2.1 Transform Type Selection
+         * If variable-sized transform coding is not enabled,
+         * then the 8x8 transform shall be used for all blocks.
+         * it is also MFX_VC1_PIC_STATE requirement.
+         */
+        if (pic_param->transform_fields.bits.variable_sized_transform_flag == 0) {
+            pic_param->transform_fields.bits.mb_level_transform_type_flag   = 1;
+            pic_param->transform_fields.bits.frame_level_transform_type     = 0;
+        }
+    }

     if (picture_type == GEN7_VC1_B_PICTURE) {
--
1.7.9.5

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/libva/attachments/20120924/e6e68be3/attachment.html>