[gst-devel] nogotiating h264 packing formats

[Wim Taymans]

On Tue, 2008-12-02 at 16:31 -0500, Olivier Crête wrote:
> On Tue, 2008-12-02 at 17:15 +0100, Wim Taymans wrote:
> > On Tue, 2008-12-02 at 16:10 +0200, Stefan Kost wrote:
> > > Wim Taymans schrieb:
> > > > On Tue, 2008-12-02 at 10:06 +0200, Stefan Kost wrote:
> > > >   
> > > >> hi,
> > > >>
> > > >> h264 streams come in two flavours:
> > > >> 1) length prefixed, each NALU is prefixed by a 32bit length field
> > > >> (length of NALU)
> > > >> 2) startcode delimmited, as described in annex b of h264 spec, NALUs are
> > > >> separated by 24 bit value 0x000001.
> > > >>
> > > >> It looks like MP4/3GP/MOV and MKV use the AVC1 encoding
> > > >> (length-prefix),while AVI, ES, and TS use
> > > >> the Annex B encoding (startcodes).
> > > >>
> > > >> When demuxing videos that contain h264 streams, demuxer will just demux
> > > >> what the file contains. decoders can check the presence of the
> > > >> codec-data on the caps. If codec-data is given, stream is length
> > > >> prefixed (1), otherwise its startedcode delimmited (where the SPS and
> > > >> PPS are in the stream an not in codec-data).
> > > >>
> > > >> When muxing we have no mean to negotiate that right now. The x264enc
> > > >> encoder has a byte-stream property to select the flavor, but that
> > > >> requires manual setup to have it right. Doing t wrong usualy still
> > > >> works, but then the file is not well playable.
> > > >>
> > > >> Should we add h264packing={length-prefixed,startcode-delimmited} to the
> > > >> caps?
> > > >>     
> > > >
> > > > codec_data on the caps currently signals the AVC1 encoding because you
> > > > need to parse the codec data to get the size of the length prefix.
> > > >
> > > > The absense of codec_data signals an annex B stream.
> > > >
> > > > You would likely build your element to handle both cases. See
> > > > rtph264pay/depay to see how this works.
> > > >
> > > > Wim
> > > >   
> > > Its quite clear for playback. I am concerned with the other way around.
> > > h264pay has "scan-mode" property, but right now only seems to implement
> > > startcode based packing. So how does h264pay tell the decoder to not
> > > send a length-prefixed stream, but a startedcode delimmited one?
> > 
> > It can't do that right now, I guess explicitly adding a property for
> > that would be the only solution then. We still have to try to make the
> > decoders use the codec_data property to detect AVC or bytestream formats
> > for backwards compatibility.
> 
> For RTP sending, there are a lot more properties that need to be
> negotiated (see RFC 3984 Section 8). To make it work properly, we'd need
> all of these to be specifiable in the caps so that the encoders can be
> setup using these.
> 
> I don't think adding properties on the encoders is the way to go here,
> since it means that they can't be negotiated. 

I meant adding a caps property so I agree. Usually the mime type
definitions for the RTP RFCs have a good set of properties that we
should be able to add to our caps. It can usually also be made
optionally because most of these properties are either optional or have
default values. 

Wim

> 
> I think all of these can be added on top of the codec_data (since
> codec-data is data that goes downstream while the other properties are
> mostly meant to upstream). In the worst case, I guess we can just invent
> a new caps (video/xx-h264 or whatever) with different parameters.
> 
> There are the same kind of things on H263-1998 (RFC 4629).
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________ gstreamer-devel mailing list gstreamer-devel at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/gstreamer-devel