[gst-devel] addition of a buffer flag

[Ronald Bultje]

Hi,

On Wed, 12 May 2004, Thomas Vander Stichele wrote:
> On Wed, 2004-05-12 at 16:14, Ronald Bultje wrote:
> > can't the mux rsimply extract this data from the packets directly?
> Not if the muxer is supposed to be codec-agnostic.

That is an ideal situation that doesn't work for any of the modern
formats (unfortunately). Matroska fails horribly here, as does Ogg. Nut,
interestingly, is fairly codec agnostic, as are AVI and MPEG.

It is not up to us to fix those problems in formats. We've never done that
for any of the previous formats, so why would we suddenly change opinion
for Ogg specifically? I'd rather use either one or the other. If we do the
'header as part of data steam', then we need to do that for any format,
including Sorensen, WMA, WMV, MPEG-4, and so on. If we don't, then we
should put codec initialization data (or "headerdata") in the caps, which
is what we currently do for all of the before-mentioned formats.

> > Note
> > that other codecs use the GstCaps property 'codec_data' for this.
> Hm, that sounds like a bad way to do that, especially given that the
> header is actually something that needs to end up in the stream as well
> to work well.  Ie, consider the case where you encode to a file; you
> don't have to write special code to make it work, it just works.
>
> Adding a caps property that really is a blob of binary data seems like a
> bad solution to this problem.  How is your codec_data serialized to a
> string when it's part of GstCaps ? can you give me an example ?

I agree that it's not nice, but it's better than nothing. And it prevents
problems in seeking in a stream through protocol. Imagine the situation
where I seek to halfway an Ogg file and only then start reading it.
Where's my header packets? At the beginning! So I would have to make the
whole datastream (protocol) format-specific to work around this mess.
Rather, I'd like to propose to put all this header-data, that is *always*
needed to start decoding and is *always* the first packet to come in, to
be first by *definition*. An easy way of doing it (and, indeed, rather
hacky in a way) is by putting it in the caps rather than in the stream
data. We define it that way because we put the data in a GstCaps, and
therefore we do not depend on a specific data order in the stream. I see
no reason to step off this method specifically for Ogg.

Now the data order thing that I mention here is not a *practical* problem.
Rather, it is an *architectural* problem and we currently don't have a
good solution to that. Therefore, our current Vorbis data protocol is
using the one solution and all other codecs use the other. I'd like to
simply pick one or the other and be done with it. I don't like a mix of
several solutions (sometimes the one, sometimes the other) for one
problem.

Serialization is simple, look at the output of qtdemux when decoding an
Sorensen-3 file or a MPEG-4 file. We already use that. It's simply
hexcodes, and the length of the data is strlen(hexstring)/2.

(e.g. caps: video/x-theora,width=(int)384,height=(int)288,
codec_data=(buffer)fe56ba3c810c).

Of course,I'm open to suggestions from others if they see different
(better) solutions to the problem drawn above.

Ronald