[gst-devel] addition of a buffer flag

[Thomas Vander Stichele]

Hi,


> That is an ideal situation that doesn't work for any of the modern
> formats (unfortunately). Matroska fails horribly here, as does Ogg. Nut,
> interestingly, is fairly codec agnostic, as are AVI and MPEG.

Ogg doesn't fail here at all.  Where does it fail ? Our implementation
of it fails.




> If we do the
> 'header as part of data steam', then we need to do that for any format,
> including Sorensen, WMA, WMV, MPEG-4, and so on. If we don't, then we
> should put codec initialization data (or "headerdata") in the caps, which
> is what we currently do for all of the before-mentioned formats.

I didn't realize that, and it sounds like a big hack to do it that way
:)

> > > Note
> > > that other codecs use the GstCaps property 'codec_data' for this.
> > Hm, that sounds like a bad way to do that, especially given that the
> > header is actually something that needs to end up in the stream as well
> > to work well.  Ie, consider the case where you encode to a file; you
> > don't have to write special code to make it work, it just works.
> >
> > Adding a caps property that really is a blob of binary data seems like a
> > bad solution to this problem.  How is your codec_data serialized to a
> > string when it's part of GstCaps ? can you give me an example ?
> 
> I agree that it's not nice, but it's better than nothing. 

Which is why, now that we've run into this problem, we discussed it
here, and we seem to think having a flag on the buffers for that is a
lot nicer.  Especially given that the header data still is data that
needs to end up in the stream, so there's no reason to special-case it
for elements that don't need it special-cased.

Really, putting binary blobs of data in the caps is just evil and
abusing caps.  Also consider the fact that an ogg muxer needs to write
two separate pages of data with header info for *each* stream in the ogg
container.  So you would also have to invent a way of making clear where
you separate the header pages in your codec_data caps property.

Seriously, this is not what caps are supposed to be used for.


> And it prevents
> problems in seeking in a stream through protocol. Imagine the situation
> where I seek to halfway an Ogg file and only then start reading it.

you do the seek on the ogg demuxer, which will handle it properly since
the ogg demuxer knows it should get the header first.  It Just Works.

> Rather, I'd like to propose to put all this header-data, that is *always*
> needed to start decoding and is *always* the first packet to come in, to
> be first by *definition*.

For ogg, it's the two first packets.  they need to be two separate
buffers since they need to be sent out as two different ogg pages.

A flag to mark a buffer as a header is the cleanest solution.


>  An easy way of doing it (and, indeed, rather
> hacky in a way) is by putting it in the caps rather than in the stream
> data. We define it that way because we put the data in a GstCaps, and
> therefore we do not depend on a specific data order in the stream. I see
> no reason to step off this method specifically for Ogg.

I propose using the flag method for all codecs then.  It's a lot nicer
to implement, it doesn't add crap to the caps, and it works for elements
that don't have to know about the difference between
headers-in-the-stream and raw-data-in-the-stream.

> Now the data order thing that I mention here is not a *practical* problem.
> Rather, it is an *architectural* problem and we currently don't have a
> good solution to that. Therefore, our current Vorbis data protocol is
> using the one solution and all other codecs use the other.


What do we do with vorbis currently ?

>  I'd like to
> simply pick one or the other and be done with it. I don't like a mix of
> several solutions (sometimes the one, sometimes the other) for one
> problem.

I agree. But let's move away from the ugly binary-data-in-caps.  Caps
aren't for binary data.

> Serialization is simple, look at the output of qtdemux when decoding an
> Sorensen-3 file or a MPEG-4 file. We already use that. It's simply
> hexcodes, and the length of the data is strlen(hexstring)/2.
> 
> (e.g. caps: video/x-theora,width=(int)384,height=(int)288,
> codec_data=(buffer)fe56ba3c810c).

Ok, so I can read and interpret those caps, *except* for the
codec_data.  What does it tell me ?

> Of course,I'm open to suggestions from others if they see different
> (better) solutions to the problem drawn above.

Tell me how the buffer flag approach fails for you.

Thomas


Dave/Dina : future TV today ! - http://www.davedina.org/
<-*- thomas (dot) apestaart (dot) org -*->
Now I find myself redeemed
'cos no one's seen the bad in me
or been where I've been
<-*- thomas (at) apestaart (dot) org -*->
URGent, best radio on the net - 24/7 ! - http://urgent.fm/