[gst-devel] thoughts on extradata

[Ronald S. Bultje]

Hi,

On Mon, 2004-03-15 at 16:35, Benjamin Otte wrote:
> (FWIW: The Xiph formats do this, too. They require the first 3 packets before 
> they can start decoding "normal" data. The theora/vorbis elements solve this 
> by refusing seeks until this data is read.)
> 
> The questions for me: 
> - What is this extradata supposed to carry?
> - (How) does the demuxer interpret extradata?
> 
> As long as it's just some additional flags you can live with parsing it during 
> caps construction. This falls flat however if someone starts adding palettes 
> or other big data chunks to the extradata.

I don't know what it is. For MPEG audio, it's ususally only a few (2)
bytes. I think this carries channel location information (so left-front,
right-front, bass, right-rear, left-rear, center, ...) and
samplerate/channelnum. Don't know for sure. I've looked it up in the
faad sources some day, but forgot... For MPEG-4, it's 32 byte of total
randomness. Probably some important (quantization or so) table, because
the MPEG-4 video decoder in ffmpeg fails horribly without it. I don't
know for sure. SVQ3 just has flags, WMA has flags1 and flags2, both
guint32... I guess that's when it starts to become binary: I don't know
what these flags *mean*.

> I always thought it would be better to make extradata the first buffer and 
> require some specific formats to start with an extradata buffer. This falls 
> flat if extradata formats vary between container formats. It has problems with 
> marking something as extradata, too. And it might not work if extradata 
> happens more often.

I don't think it's standardized. :-(. So in many cases we'll need to
convert to some GStreamer-understandable version of extradata/initdata
anyway. Think matroska (who does *weird* things in their codec_private)
or so...

for now, I just want the field to carry the information that ffmpeg
expects. That's lame, but will bring us some better media support
short-term. Long-term, I'll look up what it all means.

Ronald