Audio/video synchronization with pre-buffered video

[Anton Protopopov]

2017-03-17 6:06 GMT-04:00 Sebastian Dröge <sebastian at centricular.com>:
> On Wed, 2017-02-15 at 12:03 -0500, Anton Protopopov wrote:
>> Hello everyone,
>>
>> I have a pipeline with live H264 video source and live AAC audio
>> source muxed together using mpegtsmux. One problem with this pipeline
>> is that the video encoder hardware can buffer some number of frames,
>> so when I am creating and setting a pipeline to PLAYING state, there
>> could be already, say, 1-2 seconds of video buffered. Audio is not
>> buffered in hardware. So when the pipeline is set to the PLAYING
>> state, video source produces a set of frames with almost the same
>> time
>> stamps and several first seconds of video are corrupted (they contain
>> old video frames with some audio frames). Is there a way to
>> synchronize such sources without dropping video frames?
>
> That sounds like you're handling timestamps wrong. What should happen
> is that buffers that belong together have the same timestamps, and if
> you have 1-2s of video before the first audio, then the audio will just
> start with a timestamps 1-2s later than the first video timestamp.

Thanks for the answer Sebastian. Currently, I solved the problem by
just dropping the buffered video (it turned out that we loose at most
1-2 frames in our situation). However, you are definitely right that I
am handling timestamps wrong. Can't you tell me how can I adjust
timestamps generated by pulseaudio? If, say, my video-source produces
buffers with timestamps starting from zero, then how can I tell
pulseaudio to generate timestamps starting from some offset X?

Anton

>
> --
> Sebastian Dröge, Centricular Ltd · http://www.centricular.com
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>