[gst-devel] muxing proposal

[Benoit Fouet]

Hi,

David Schleef wrote:
> Attempting to get Dirac muxing working correctly with most containers
> has been a difficult task recently.  GStreamer has been the easiest
> media framework *by* *far*; other frameworks like DirectShow and
> QuickTime really only allow muxing into one specific container.  (In
> some ways, gstreamer is the same way, with ogg.)
>
> The main problems, as I see it, are:
>
>  - All our muxers suck.
>
>  - Each container format has special mapping requirements for streams.
>    AVI needs special chunking of Dirac packets, MPEG-TS needs
>    presentation timestamps, Ogg needs granulepos, etc.  Right now,
>    each muxer reparses and/or attempts to reconstruct this information
>    in the muxer (but mostly just does it wrong).
>
>  - Adding a new format requires modifying the muxer.  This is the same
>    problem as convincing Apple to add Dirac to their mp4 muxer, except
>    that I'm lucky enough to have commit access here.
>
>
>   

there will still be the problem of extradata that the muxer has to write.
or do you also plan to have the new elements (or modified parsers) to
take care of the extradata too ?

> What I propose is this:
>
>  - Muxers should be modified to implement a native stream format, e.g.,
>    video/x-avi-part for avimux, with a fourcc describing the stream.
>    Muxers may optionally accept other formats as they do now, but are
>    responsible for parsing and discovering values that the container
>    needs to store.  These streams have a well-defined structure, with
>    the intention that buffers are directly encapsulated into output
>    packets.
>    
>   

Doesn't that mean code duplication in multiple plugins ?

>  - For each of these muxer substream formats, define additional
>    stream-specific buffer flags as necessary and override
>    GST_BUFFER_OFFSET_END() for stream- specific purposes (like granulepos.)
>
>  - A set of new elements (or mods to parsers) that parse encoded streams
>    into the correct format for muxers.  Thus, diracparseforavi would
>    parse a dirac stream into the correct chunks to go into AVI, as well
>    as setting the key frame flags correctly.  (Key frame flags in AVI and
>    QuickTime are more restricted than MPEG or Ogg.)  diracparseforogg
>    would add Ogg granulepos to each buffer.
>
>    (I'm indifferent to using separate elements or a single diracparse
>    element that can output many parsed formats.)
>
>  - Theoretically, this should be backward compatible.  We can move toward
>    using the native stream formats gradually.
>
>  - Ogg will still require stream-specific code in the muxer.
>
> Once parsers are responsible for delivering buffers to the muxer that
> the muxer understands, I'm hoping that writing a muxer becomes a lot
> closer to "while(1) { collect a buffer; encapsulate; push }".
>
>   

if one is willing to support a specific stream encapsulation in a
specific container, I don't see why (s)he would not do it directly in
the muxer
(but I might have overlooked something)

-- 
Benoit Fouet
Purple Labs S.A.
www.purplelabs.com