correct way to combine audio and video stream by frame timestamp

nucleus_wind felixjojo199 at
Sat Nov 24 07:52:04 UTC 2018


This is what I'm trying to do:

audio stream ----->\
                              -----> appsink -> custom code to package audio
video into mkv
video stream ----->/

The audio, video stream could be coming from live sources or demuxed from
files. I know that appsink only has one sink pad. Right now I wrote two
on_new_sample callbacks and created two independent pipelines, one for audio
and one for video. But then I ran into issue where audio and video frames
can be out of order, so I have to write custom logic to sync the frames. I
feel that this should not be a rare use case. I know there is matroskamux
but it doesnt fit my use case. I need to get the buffers in timestamp order.
I did some research and found that there exists elements like input-select
and multiqueue, but I cant figure out how to use them correctly here or if
it is possible at all. Any help is greatly appreciated.

Sent from:

More information about the gstreamer-devel mailing list