[gst-devel] mpeg2dec plugin development notes

[David I. Lehn]

I was asked to write up some notes about developing the mpeg2dec plugin.
Hopefully something is gained out of this for future plugin developers.

If I start to not make sense maybe you'll need to look at the code to
understand what I'm saying ;)

First thing is that you have to understand how mpeg2 processing works
and how libmpeg2 handles it.  The decoder does not just take input data
and produce output frames.  Well, it does.  Just not in an easy way.
MPEG2 uses predictive frames (look this up elsewhere if you need to
know...).  libmpeg2 is designed such that you allocate frames, then get
asked to return frames (with a flag specifying predictive frames that
will be used later), and whenever the decoder has enough data it calls a
draw function for a frame.  The order that you return frames to the
decoder is not the same that they are actually drawn. Sorry if that's
confusing...

So after I wrapped my head around that stuff had to figure out how to
best make a plugin.  Well, first I just took the broken first attempt
for a plugin.  That's a good bit of starter code.  Then I copied and
modified other stuff like factories and caps info from the current
mpeg2play plugin.  That's most of the boilerplate code.

This plugin is perfect for a loop based processing.  Really easy in
fact.  Just stay in a loop: read buffer, tell libmpeg2 to process.

But then we hit the problem of how to pass data on.  Well, in the draw
function you know you can pass that frame down the line.  But it may
still be used by the decoder as a predictive frame.  So I just marked it
readonly.  But readonly is just a buffer flag.  It would be nice to
unmark it readonly.  But if another element tries to mark it readonly
too for some reason... well, we would need a readonly refcount for that.
I'm not sure how much this matters in practice.

Another problem I hit is memory management.  If you were just
immediately using the frame data in the drawing call you could just
allocate 3 frame buffers and reuse them.  But gstreamer requires a
stream of output.  In general you would use a GMemChunk for this along
with buffer pool.  Let it manage efficient buffer reuse.  (I have no
idea how you choose a 'good' value for how many atoms go in that
chunk??)  But libmpeg2 could, based on input stream, call a setup
callback changing the output frame size.  That makes the buffering kind
of a mess.  You would need a new mem chunk for the new frames size.  Now
you would need to manage refcounting on GMemChunk (damnit, why doesnt
glib do this?).

So the current code is rather inefficient.  It mallocs every new frame
and deallocs it when it's no longer needed.  Yuck.

One way to handle some of this buffering is to only push buffers that
libmpeg2 is done with.  This eliminates the readonly issues.  Probably
just changes the mem management into a slightly different problem.  But
it does mean that there will be a higher latency and more mem usage in
the plugin since you won't be sharing some frames with other elements.

That's also another issue... the plugin is doing the memory management
itself.  At the moment there doesn't seem to be functionality for an
element to ask other elements for a more efficient way to handle
buffers.  I don't know what is required for this.

That all I can think of for now.  Overall not too bad on the gstreamer
end.  All the heavy lifting is done by libmpeg2:
http://www.linuxvideo.org/mpeg2dec/

-dave
-- 
David I. Lehn <dlehn at vt.edu>  | http://www.lehn.org/~dlehn/
Computer Engineering Graduate @ Virginia Tech in sunny Blacksburg, VA