How to sync audio and video when playing back samples generated by appsink?

[phdloli]

This problem is a little special. Let me describe it in detail.

Now I have producer part and consumer part. 
The producer keeps generating video & audio samples from MP4 file. 
The consumer keeps playing back video & audio samples after receiving them.
The producer part is like this 
(1)"filesrc -> qtdemux -> queue ->h264parse -> appsink(video) -> sending to
consumer sample by sample
                                 -> queue -> aacparse  -> appsink(audio) ->
sending to consumer sample by sample"
(2)I use " g_signal_emit_by_name (appsink, "pull-sample", &sample) " to get
video or audio sample.
(3)I will also record the sample caps by using "caps =
gst_sample_get_caps(sample);" I need this 
information to set up consumer.

The consumer part is like this 
(1)"receiving sample by sample -> appsrc(video) -> queue -> avdec_h264 ->
autovideosink
     receiving sample by sample -> appsrc(audio) -> queue -> faad ->
autoaudiosink"
(2)I'm using Push-Mode of appsrc. Like this
    "g_signal_connect (appsrc, "need-data", G_CALLBACK (start_feed), audio);
     g_signal_connect (appsrc, "enough-data", G_CALLBACK (stop_feed),
audio);"
(3)Before consuming video and audio sample by sample, the consumer will
first set up the pipeline using the video and audio caps information
separately.

==================================================================

Here is my problem.

For one mp4 file, this procedure works perfect. But for another mp4 file,
the audio and video are just out of sync. 

*Actually, I don't quite understand how the consumer part sync the video and
audio. Although I add all the plugins into one pipeline. I link video
plugins and audio plugins separately. 
(1)Do the samples which appsink generated contain the synchronization
information? 
(2)And what else I should do in the consumer part if I want sync the video
samples and audio sample? 
(3)And what's the relationship between GstSample number and the video
framerate and audio samplerate?*

==========================================================
Here are more information about the example MP4 file. 
(1)The caps of the file which works well:
"Video streaminfo -----
video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)3, profile=(string)high,
codec_data=(buffer)0164001effe1001a6764001eacc8602a0c7e4c0440000003004000000ca3c58b678001000568e9bb2c8b,
width=(int)672, height=(int)378, framerate=(fraction)25/1,
pixel-aspect-ratio=(fraction)1/1, parsed=(boolean)true
Audio streaminfo -----
audio/mpeg, mpegversion=(int)4, framed=(boolean)true,
stream-format=(string)raw, level=(string)1, base-profile=(string)lc,
profile=(string)lc, codec_data=(buffer)139056e5a54800, rate=(int)22050,
channels=(int)2"

The length of this mp4 file is about 10 min.

(2) The caps of the file which out of sync.
"Video streaminfo -----
video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)2.1, profile=(string)high,
codec_data=(buffer)01640015ffe1001a67640015acd941410fcb8088000003000800000301e4f8a14cb001000468febcb0,
width=(int)320, height=(int)240, framerate=(fraction)2000/141,
pixel-aspect-ratio=(fraction)1/1, parsed=(boolean)true
Audio streaminfo -----
audio/mpeg, mpegversion=(int)4, framed=(boolean)true,
stream-format=(string)raw, level=(string)2, base-profile=(string)lc,
profile=(string)lc, codec_data=(buffer)1208, rate=(int)44100,
channels=(int)1"

The length of this mp4 file is about 15 sec.

My platform is Mac OS, and Gstreamer is 1.4.3.

I really don't know how to handle the synchronization issue. I'm a newbie of
Gstreamer. Hope I describe my situation clearly... If you have any idea,
please tell me.
Thank you very much!







--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/How-to-sync-audio-and-video-when-playing-back-samples-generated-by-appsink-tp4670093.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.