Synchronization of external events with audio

[jcc273]

Hello,

I am relatively new to gstreamer and started working with it over the last
few months.  For a project I am working on i have an audio stream (from file
or live) that contains data on one channel and audio on the other.  I wrote
a custom plugin that decodes the data channel (AMI) and then duplicates the
audio channel over the data so that left and right have the same audio and
sends the buffer on.  Afterwards i have a volume plugin, equalizer, and then
alsasink.  My sources are either alsasrc (line-in) or filesrc.

Now for the data I decode I need to spawn an external event to happen at the
time when the audio will occur.  So what i do is try and calculate when the
event should occur based on the current time, buffer time, sample number in
current buffer, and the base time.  This gets me close, especially when
playing a file and not changing the playback rate at at all:

now = gst_clock_get_time(element->clock)/1000000;
eventTime = (gst_element_get_base_time(element)/1000000) +
(((GST_BUFFER_TIMESTAMP(buf)/1000000) +
(sampleInBuffer/SAMPLE_RATE_MS))/playRate);
msToEvent = eventTime-now;


2 things really cause synchronization of audio and event to be off though:

1.  Live Source (alsasrc) - This causes syncronization to really be off over
time.  I am thinking i need to handle latency somehow?  I also had to add
sync=false to alsasink in live streams or the audio would just stop playing
after about 1 second.  Maybe this param is causing issues?  But the audio
will not play at all without it!  Here is my live pipeline:  filesrc
location=XXX ! mad ! audiocovnert ! CustomDecoder ! equalizer-3bands !
volume ! alsasink

2.  Changing playback rate (via seek) - It is sometimes desired to play a
file back at a slower or faster rate.  When doing this my calculation above
seems to correctly modify the event speeds, but the event and audio or not
in sync, they seem to be consistently off by some constant amount, like
changing the rate ads some constant delay to the audio play time?  Here is
my file pipeline:  alsasrc ! audiocovnert ! CustomDecoder ! equalizer-3bands
! volume ! alsasink sync=false

Maybe I am going about this the wrong way all together, but any advice as to
how i should accomplish this synchronization would be greatly appreciated!!! 
I have read through all the gstreamer clock documentation and i thought i
was understanding it all correctly, but i am still not able to achieve
correct synchronization : /.

Thanks,
Jarrod



--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/Synchronization-of-external-events-with-audio-tp4672202.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.