[Mesa-dev] [PATCH] vl/rbsp: add a check for emulation prevention three byte

[Leo Liu]

On 08/09/2016 04:26 AM, Christian König wrote:
> Am 08.08.2016 um 22:10 schrieb Leo Liu:
>> This is the case when the "00 00 03" is very close to the beginning of
>> nal unit header
>
> I see where the problem is, but the fix is incorrect.
>
> You always search for the emulation prevention three byte even when 
> the previous fill has done so already. So it could happen in theory 
> that you revert the escaping twice, e.g. remove valid bits.

Right. Will send v2.

Thanks,
Leo

>
> You need to add this extra check to the end of vl_rbsp_init() after we 
> searched for the end of the NAL unit.
>
> Regards,
> Christian.
>
>>
>> Signed-off-by: Leo Liu <leo.liu at amd.com>
>> ---
>>   src/gallium/auxiliary/vl/vl_rbsp.h | 10 +++++++++-
>>   1 file changed, 9 insertions(+), 1 deletion(-)
>>
>> diff --git a/src/gallium/auxiliary/vl/vl_rbsp.h 
>> b/src/gallium/auxiliary/vl/vl_rbsp.h
>> index 7867238..c134d31 100644
>> --- a/src/gallium/auxiliary/vl/vl_rbsp.h
>> +++ b/src/gallium/auxiliary/vl/vl_rbsp.h
>> @@ -77,8 +77,16 @@ static inline void vl_rbsp_fillbits(struct vl_rbsp 
>> *rbsp)
>>      unsigned i, bits;
>>        /* abort if we still have enough bits */
>> -   if (valid >= 32)
>> +   if (valid >= 32) {
>> +      /* search for the emulation prevention three byte */
>> +      for (i = 24; i <= valid; i += 8) {
>> +         if ((vl_vlc_peekbits(&rbsp->nal, i) & 0xffffff) == 0x3) {
>> +            vl_vlc_removebits(&rbsp->nal, i - 8, 8);
>> +            i += 8;
>> +         }
>> +      }
>>         return;
>> +   }
>>        vl_vlc_fillbits(&rbsp->nal);
>
>