[gst-devel] Mixer plugin

[Simon Per Soren Kagedal]

Hi,

I'm writing a mixer plugin for GStreamer, and I've run into some
issues that I would like to ask for ideas about.  

Basic Idea: A filter that takes m input audio streams (sinks) and
mixes to n sources, by adding together the data.  For simplicity, I'll
assume here that all streams are mono audio/raw data of the same
format.  Then, to mix stereo data, one would have to use channel
splitters/mergers.  It would be convinient though to let all in/out
streams have any number of channels, and probably more efficient.

Problems/Issues:

1. Setting "Panning" Values

For each sink i=1..m, you should be able to set (and get) how much of
it's signal should be sent to each source j=1..n.

So how do I do this?  gtk_object_set/get is the only interface I know
of for setting properties of elements in a plugin, and as far as I
know, there is no way to access "subproperties" except by registering
one argument type for every g_strdup_printf("mix%d%d",i,j).  This idea
doesn't appeal to me very much.  I would like both the sinks and the
sources to be "hotpluggable", and then you would have to add and
remove lots of argument types when streams come and go.

So..  any other ideas or is this the way I have to go?  

One twist is that you might would want to like a volume change to be
described as time-stamped sample accurate event.

2. Loop Based

The mixer uses a loopfunc() to do its work.  I could not think of or
make work any other way.  Correct me if I'm wrong, but it seems to me
that a loopfunc-based implentation should only be used if neccessary?
For example, I get constant audio glitches if I use several loopbased
elements in one pipeline, such as:

gstreamer-launch disksrc location=startup.raw ! identity loop_based=t ! identity loop_based=t ! audiosink

Is this is a bug, is there a workaround, or is this case just stupid?

(yeah, I know that's an ugly hack for setting the boolean value, but
it works. :) any chance gst_parse_launch will recognize other types
than strings soon?)

So is there any way I can do it without a loop function?  Can I have a
chain function that only pushes when the data from all the sinks has
been received?  Or could I define some sort of "pull" function on the
sinks that would in turn pull it's data from the sources?  I tried
using a get function but that didn't work.  (for this approach, you
can't have n source streams, but that doesn't really matter that much,
it could just always be one src with n channels, and then you'd use
some channel splitter afterwards if neccessary.)

If there is anywhere an explanation (except the source code :) ) of
how exactly the scheduling works, I would be very interested.

Another question: the manual says "chain based elements receive a
buffer of data and are supposed to handle the data and perform a
gst_pad_push".  What does _A_ gst_pad_push mean here?  Not
neccessarily 1, since GstTee pushes to all it's outsrces.  But can it
be zero?

3. Channels

For maximum flexibility, I would like to let the user have any number
of channels of any of the sinks and srces.  For sinks it is easy, the
channel number comes at meta info on the stream.  Some oddities
though..  User wants to set mixing values for a specific sink before
the network is running, but then the mixer doesn't know how many
channels there are going to be on that sink, so how to store/verify
the value...?  Anyway, I don't know how to set these properties, see
problem 1.  Sorry, I'm mumbling, but what it all boils down to is... 

4. Philosophy

I've tried to do the mixer using a "hard things possible but easy
things easy" approach.  To make easy things easy, it should
automatically set mixing values (if not explicitly specified) so that
things like the following work as it ought to:

  stereosrc1 = ...
  stereosrc2 = ...
  gst_pad_connect (gst_element_get_pad (stereosrc1, "src"),
		   gst_element_request_pad_by_name (mixer, "sink%d"));
  gst_pad_connect (gst_element_get_pad (stereosrc2, "src"),
		   gst_element_request_pad_by_name (mixer, "sink%d"));
  gst_pad_connect (gst_element_request_pad_by_name (mixer, "src%d"),
		   gst_element_get_pad (audiosink, "sink"));

But it does kind of complicate things, and I think the #1 philosophy
should be "do all things efficient".  So is it still a good idea to
design for the hard cases and try to optimize afterwards, or would be
better to have like "monomixer" and "stereomixer" etc. in addition to
a more general mixer? 

I hope someone has time to comment on this, it would really help me
get into the GStreamer framework.  Especially problem 1.  I'd post my
source code, but it's right now in the middle of a redesign that I'm
not really satisfied with, so...

Thanks.
Simon.