[Libva] [PATCH] libva-intel-driver: fix i965 encoder wrong bits shift operation

[Sean V Kelley]

On Tue, Jan 17, 2017 at 6:48 PM, Xiang, Haihao <haihao.xiang at intel.com>
wrote:

> On Tue, 2017-01-17 at 13:15 -0800, Sean V Kelley wrote:
> > From: Kuang-che Wu <kcwu at chromium.org>
> >
> > shift uint32_t by 32 bits is undefined behavior.
> >
> > For this particular case: when invoke avc_bitstream_put_ui() with 32
> > bits value at byte position of multiple of 4, existing 32 bits garbage
> > data in the buffer may be retained instead of cleared. The result is,
> > the position of NALU start code (0x00000001) looks like overwritten by
> > garbage value.
> >
> > Patch has been tested and used upstream:
> > https://chromium-review.googlesource.com/#/c/410541/
> >
> > Signed-off-by: Kuang-che Wu <kcwu at chromium.org>
> > Signed-off-by: Sean V Kelley <seanvk at posteo.de>
> > ---
> >  src/i965_encoder_utils.c | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/src/i965_encoder_utils.c b/src/i965_encoder_utils.c
> > index ac58cd1a..e061d071 100644
> > --- a/src/i965_encoder_utils.c
> > +++ b/src/i965_encoder_utils.c
> > @@ -134,7 +134,11 @@ avc_bitstream_put_ui(avc_bitstream *bs, unsigned
> int val, int size_in_bits)
> >          bs->buffer[pos] = (bs->buffer[pos] << size_in_bits | val);
> >      } else {
> >          size_in_bits -= bit_left;
> > -        bs->buffer[pos] = (bs->buffer[pos] << bit_left) | (val >>
> size_in_bits);
> > +        if (bit_left == 32) {
> > +            bs->buffer[pos] = (val >> size_in_bits);
>
> The input value of size_in_bits should be less than or equal to 32, so now
> the
> value of size_in_bits is 0 for this case, I think  "bs->buffer[pos] = val"
> is
> more clear.
>

That's a reasonable change.  Thanks Haihao.  I'll update the patch.

Sean


>
> > +        } else {
> > +            bs->buffer[pos] = (bs->buffer[pos] << bit_left) | (val >>
> size_in_bits);
> > +        }
> >          bs->buffer[pos] = swap32(bs->buffer[pos]);
> >
> >          if (pos + 1 == bs->max_size_in_dword) {
> _______________________________________________
> Libva mailing list
> Libva at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/libva
>



-- 
Sean V. Kelley <sean.v.kelley at intel.com>
Open Source Technology Center / SSG
Intel Corp.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libva/attachments/20170118/ab224014/attachment.html>