[gst-devel] Game-style audio

[Steve Fink]

[apologies if this has been asked before. I cannot access the list archives.]

I'm looking at using gstreamer for managing the audio and video of my
application, and so far I'm really liking it. Everything works about
the way I would hope that it would. But I'm wondering how well-suited
it is to non-streaming audio, as you'd find in (for example) a game.
My application is not quite a game, most of the time, but it has
similar needs.

Specifically, I want to be able to fire off up to a dozen or two
simultaneous short audio clips. I want to be able to:

 - receive a notification when each individual clip is done
 - be able to dynamically envelope the volume and possibly other
properties of each clip, separately
 - when a new audio clip is fired and there are already N playing, I
want to intelligently choose another clip to kill off. That will
normally be the oldest, but I might want to group them so that a new
clip kills the oldest within its group only.
 - be able to query the current progress of each sound (or at least
receive events when various tags within the clips are reached), and be
able to associate it with the appropriate sound

gstreamer seems great for doing all of the above for one or a small
fixed number of simultaneous audio streams, but I'm unclear as to the
best approach for managing a varying number of clips all firing off at
different times. Should I:

 1. Do it all externally to gstreamer, and produce a single pre-mixed
audio stream?
 2. Create/destroy a gstreamer pipeline for each instance?
 3. Keep some fixed maximum number of pipelines alive and rolling at
all times, and let them play silence when they're not being used?
 4. Some combination or other option?

#3 seems the nicest, but I can't use gstreamer's end-of-stream events,
and I have to keep a bunch of processing pipelines alive and using
resources all the time even when I'm not playing anything. I need
low-latency sound (it's triggered in response to user events, and
cannot be predicted in advance) -- would it be fast enough to keep the
silent pipelines paused when inactive, and start them playing when the
trigger happens? I'll probably try it to see, but I'm wondering if
state changes are generally regarded as a bad idea from a latency
perspective.

At some point, I'd also like to try out some dynamic mixing volume
control (eg if an important clip is triggered, then drop everything
else down to make sure it's heard, then gradually bring them back up
again). What plugins should I be looking at for this or any other
part?

Thanks,
Steve