[Mesa-dev] [PATCH 1/2] st/vdpau: add VC-1 startcode if none is found in the stream

[Emeric Grange]

2012/3/15 Christian König <deathsimple at vodafone.de>

> Hi Emeric,
>
> On 15.03.2012 15:13, Emeric Grange wrote:
>
>> I actually made some code that does exactly the opposite, for H.264 and
>> VP8, removing the start code from the first buffer containing it (so the
>> first buffer alltoghether or just the 3 first bytes of the first buffer). I
>> think the start codes, at least for H..264 are just here so the decoding
>> application (in this case the state tracker) can make sure the buffers are
>> valid video buffers, and are not used in the decoding process.
>>
> At least for H264 and AMD hardware that isn't such a good idea. The
> startcodes are the indicators for the hardware decoder where slices
> actually starts and ends (hence the name), so if the decoder doesn't sees a
> startcode at the beginning of the buffer it will just skip forward until it
> finds something that looks valid. And if I understood Maarten Lankhorst
> correctly the nvidia hardware is doing this in a very similar way.


All right, if the hardware decoders are using the startcodes that way.

 In my VP8 implementation I require a 0x9D012A start code, similar to H.264
>> that requires a 0x000001. I asked on the VDPAU mailing list if it was a
>> good idea, and what was the purpose of these start code in the first place
>> but I did not get an answer.
>>
> The idea behind startcodes is to have something that doesn't normally
> occur in the encoded bitstream, so you can do network/transport
> synchronization with it. In most of the modern codecs (MPEG2, MPEG4, H264,
> VC-1) there seems to be some kind of consensus to use the bytes 0x00 0x00
> 0x01 + some code for that, but it could actually be any unique combination
> of bytes.
>
> So when a network error occur (for example cause it is a satellite, DVB,
> whatever wireless broadcast transmission) the decoder can always fallback
> to search for the next start code and so resynchronize the stream.


As far as I know only MPEG TS uses the 0x000001 start code for
network synchronization purpose as you said (I'm not even sure it's really
for resync). The H.264 Annex B is also the only "container format" that
uses these start codes. MKV or MP4 and probably every other modern
container formats doesn't uses start codes.

The H.264 specification does forbid 0x000000 to 0x000003 byte
pattern inside NAL Units, so that they can be used
as synchronization pattern, and says that these codes must be replaced
by 0x00000300 to 0x00000303. But as far as I know (and except from Annex B)
these codes are only used at higher levels, not by the codec itself.

 However I have no idea if a VC1 decoder requires the presence of start
>> codes, and your implementation is fine by me, I can still hook a
>> vlVdpDecoderRemoveVP8Startcode**() function similar to
>> vlVdpDecoderFixVC1Startcode().
>>
> Well, the problem with VC-1 is that certain container formats (IIRC WMV)
> doesn't use start codes, so they are not suited for streaming/broadcast
> transmission, but on the other hand safes 4 bytes of space for each frame
> (yeah another great Microsoft invention). So since the hardware decoders
> seem to need the start codes we must add them manually.
>

Well the start codes are not required by the video formats themselves,
that's why I was supposing the hardware decoders will not use them.

As example, when mplayer decode a H.264 stream through VDPAU it just
hardcode the start code into a separate buffer, because it will not find
that code into the H.264 bitstream. So what is the difference with the WMV
case ? I think you should extend your mechanism (with a different start
code however) to H.264 as well.

Honestly I have no idea what to do in the VP8 case, so do whatever you
> think is right there.


The VP8 "uncompressed frame header" requires the presence of a 0x9D012A
start code for key frames, so let's just use this one as mandatory VDPAU
start code. This is a weird case because the start code is only present in
key frames, and not even at the start of the frame header. Besides who
stream VP8 not through webm files and tcp networks ? Seems like a wast of
space to me...

Regards,
Emeric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/mesa-dev/attachments/20120315/5ec47467/attachment.htm>