[gst-devel] muxing proposal

[David Schleef]

Attempting to get Dirac muxing working correctly with most containers
has been a difficult task recently.  GStreamer has been the easiest
media framework *by* *far*; other frameworks like DirectShow and
QuickTime really only allow muxing into one specific container.  (In
some ways, gstreamer is the same way, with ogg.)

The main problems, as I see it, are:

 - All our muxers suck.

 - Each container format has special mapping requirements for streams.
   AVI needs special chunking of Dirac packets, MPEG-TS needs
   presentation timestamps, Ogg needs granulepos, etc.  Right now,
   each muxer reparses and/or attempts to reconstruct this information
   in the muxer (but mostly just does it wrong).

 - Adding a new format requires modifying the muxer.  This is the same
   problem as convincing Apple to add Dirac to their mp4 muxer, except
   that I'm lucky enough to have commit access here.


What I propose is this:

 - Muxers should be modified to implement a native stream format, e.g.,
   video/x-avi-part for avimux, with a fourcc describing the stream.
   Muxers may optionally accept other formats as they do now, but are
   responsible for parsing and discovering values that the container
   needs to store.  These streams have a well-defined structure, with
   the intention that buffers are directly encapsulated into output
   packets.
   
 - For each of these muxer substream formats, define additional
   stream-specific buffer flags as necessary and override
   GST_BUFFER_OFFSET_END() for stream- specific purposes (like granulepos.)

 - A set of new elements (or mods to parsers) that parse encoded streams
   into the correct format for muxers.  Thus, diracparseforavi would
   parse a dirac stream into the correct chunks to go into AVI, as well
   as setting the key frame flags correctly.  (Key frame flags in AVI and
   QuickTime are more restricted than MPEG or Ogg.)  diracparseforogg
   would add Ogg granulepos to each buffer.

   (I'm indifferent to using separate elements or a single diracparse
   element that can output many parsed formats.)

 - Theoretically, this should be backward compatible.  We can move toward
   using the native stream formats gradually.

 - Ogg will still require stream-specific code in the muxer.

Once parsers are responsible for delivering buffers to the muxer that
the muxer understands, I'm hoping that writing a muxer becomes a lot
closer to "while(1) { collect a buffer; encapsulate; push }".



dave...