Automatic time alignment by signal analysis

[Sean McNamara]

Hi,

On Wed, Apr 20, 2011 at 12:50 PM, Benjamin M. Schwartz
<bmschwar at fas.harvard.edu> wrote:
> I often find myself wanting to edit multiple audio streams that were
> recorded by separate devices.  For example, I might have a high quality
> audio recorder, and a separate consumer videocamera with a low-quality
> microphone.  The two devices, being separate, have different start times
> and slightly different clock rates.
>
> I would like to align these two tracks automatically based on analysis of
> the audio signals' contents.  Does anyone know of a tool that does this?
> If not, I am considering writing such a program, as I am interested in the
> signal processing problem.
>
> I would like to make this tool as friendly to gstreamer as possible, so
> that it can be used in gst-world applications like PiTiVi.  However, I do
> not know how to structure the program.  The program must perform analysis
> on the entire duration of two different audio streams (potentially
> multiple hours or longer) before determining the required offset and
> stretch factor.  These parameters then need to be applied, possibly by
> editing a gnonlin timeline.
>
> How should I write such a program ... or have you already written it?
>
> --Ben

It may not be what you're looking for (no integration with Gstreamer),
but PulseAudio has code in "module-combine" to:

1. Combine two separate sinks into one virtual sink. Example: two sound cards.
2. Dynamically calculate the timing deviation, on the fly, and adapt
to keep the two sinks in sync by resampling one of them
3. Do software mixing on the two streams to combine them into one
stream that then becomes the output stream of the virtual sink.

Sounds like you may not want 1. or 3. but just the functionality in 2.
Well, PulseAudio is open source, so if you're just looking for some
algorithmic inspiration on how to do this, check out the source of PA:
http://git.0pointer.de/?p=pulseaudio.git;a=blob;f=src/modules/module-combine.c;h=ab93c05efebf77236c2204a6765f21be6ee92f92;hb=HEAD

Have a look at the adjust_rates() functionality in particular, but
also the contexts in which that function is called.

HTH,

Sean

>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>