[gst-devel] nogotiating h264 packing formats

[Olivier Crête]

On Tue, 2008-12-02 at 17:15 +0100, Wim Taymans wrote:
> On Tue, 2008-12-02 at 16:10 +0200, Stefan Kost wrote:
> > Wim Taymans schrieb:
> > > On Tue, 2008-12-02 at 10:06 +0200, Stefan Kost wrote:
> > >   
> > >> hi,
> > >>
> > >> h264 streams come in two flavours:
> > >> 1) length prefixed, each NALU is prefixed by a 32bit length field
> > >> (length of NALU)
> > >> 2) startcode delimmited, as described in annex b of h264 spec, NALUs are
> > >> separated by 24 bit value 0x000001.
> > >>
> > >> It looks like MP4/3GP/MOV and MKV use the AVC1 encoding
> > >> (length-prefix),while AVI, ES, and TS use
> > >> the Annex B encoding (startcodes).
> > >>
> > >> When demuxing videos that contain h264 streams, demuxer will just demux
> > >> what the file contains. decoders can check the presence of the
> > >> codec-data on the caps. If codec-data is given, stream is length
> > >> prefixed (1), otherwise its startedcode delimmited (where the SPS and
> > >> PPS are in the stream an not in codec-data).
> > >>
> > >> When muxing we have no mean to negotiate that right now. The x264enc
> > >> encoder has a byte-stream property to select the flavor, but that
> > >> requires manual setup to have it right. Doing t wrong usualy still
> > >> works, but then the file is not well playable.
> > >>
> > >> Should we add h264packing={length-prefixed,startcode-delimmited} to the
> > >> caps?
> > >>     
> > >
> > > codec_data on the caps currently signals the AVC1 encoding because you
> > > need to parse the codec data to get the size of the length prefix.
> > >
> > > The absense of codec_data signals an annex B stream.
> > >
> > > You would likely build your element to handle both cases. See
> > > rtph264pay/depay to see how this works.
> > >
> > > Wim
> > >   
> > Its quite clear for playback. I am concerned with the other way around.
> > h264pay has "scan-mode" property, but right now only seems to implement
> > startcode based packing. So how does h264pay tell the decoder to not
> > send a length-prefixed stream, but a startedcode delimmited one?
> 
> It can't do that right now, I guess explicitly adding a property for
> that would be the only solution then. We still have to try to make the
> decoders use the codec_data property to detect AVC or bytestream formats
> for backwards compatibility.

For RTP sending, there are a lot more properties that need to be
negotiated (see RFC 3984 Section 8). To make it work properly, we'd need
all of these to be specifiable in the caps so that the encoders can be
setup using these.

I don't think adding properties on the encoders is the way to go here,
since it means that they can't be negotiated. 

I think all of these can be added on top of the codec_data (since
codec-data is data that goes downstream while the other properties are
mostly meant to upstream). In the worst case, I guess we can just invent
a new caps (video/xx-h264 or whatever) with different parameters.

There are the same kind of things on H263-1998 (RFC 4629).

-- 
Olivier Crête
olivier.crete at collabora.co.uk
Collabora Ltd
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20081202/03d9d0ab/attachment.pgp>