[Libva] [PATCH] vp8: fix loop filter for bitexact reconstruction.

[Gwenole Beauchesne]

FYI, I will push that in the next couple of days. Just submitting
prior to forgetting about it.

2014-04-08 14:56 GMT+02:00 Gwenole Beauchesne <gb.devel at gmail.com>:
> Each loop filter delta update value shall be encoded within 7 bits,
> including the sign bit and 6-bit magnitude in 2's complement. So,
> don't propagate the sign bit while packing the filter level values.
>
> Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne at intel.com>
> ---
>  src/gen8_mfd.c | 16 ++++++++--------
>  1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/src/gen8_mfd.c b/src/gen8_mfd.c
> index a801a7c..177a11d 100644
> --- a/src/gen8_mfd.c
> +++ b/src/gen8_mfd.c
> @@ -2942,16 +2942,16 @@ gen8_mfd_vp8_pic_state(VADriverContextP ctx,
>      }
>
>      OUT_BCS_BATCH(batch,
> -                  pic_param->loop_filter_deltas_ref_frame[3] << 24 |
> -                  pic_param->loop_filter_deltas_ref_frame[2] << 16 |
> -                  pic_param->loop_filter_deltas_ref_frame[1] <<  8 |
> -                  pic_param->loop_filter_deltas_ref_frame[0] <<  0);
> +                  (pic_param->loop_filter_deltas_ref_frame[3] & 0x7f) << 24 |
> +                  (pic_param->loop_filter_deltas_ref_frame[2] & 0x7f) << 16 |
> +                  (pic_param->loop_filter_deltas_ref_frame[1] & 0x7f) <<  8 |
> +                  (pic_param->loop_filter_deltas_ref_frame[0] & 0x7f) <<  0);
>
>      OUT_BCS_BATCH(batch,
> -                  pic_param->loop_filter_deltas_mode[3] << 24 |
> -                  pic_param->loop_filter_deltas_mode[2] << 16 |
> -                  pic_param->loop_filter_deltas_mode[1] <<  8 |
> -                  pic_param->loop_filter_deltas_mode[0] <<  0);
> +                  (pic_param->loop_filter_deltas_mode[3] & 0x7f) << 24 |
> +                  (pic_param->loop_filter_deltas_mode[2] & 0x7f) << 16 |
> +                  (pic_param->loop_filter_deltas_mode[1] & 0x7f) <<  8 |
> +                  (pic_param->loop_filter_deltas_mode[0] & 0x7f) <<  0);
>
>      /* segmentation id stream base address, DW35-DW37 */
>      OUT_BCS_BATCH(batch, 0);
> --
> 1.8.3.2
>