correct way to combine audio and video stream by frame timestamp

[nucleus_wind]

Hello,

This is what I'm trying to do:

audio stream ----->\
                              -----> appsink -> custom code to package audio
video into mkv
video stream ----->/

The audio, video stream could be coming from live sources or demuxed from
files. I know that appsink only has one sink pad. Right now I wrote two
on_new_sample callbacks and created two independent pipelines, one for audio
and one for video. But then I ran into issue where audio and video frames
can be out of order, so I have to write custom logic to sync the frames. I
feel that this should not be a rare use case. I know there is matroskamux
but it doesnt fit my use case. I need to get the buffers in timestamp order.
I did some research and found that there exists elements like input-select
and multiqueue, but I cant figure out how to use them correctly here or if
it is possible at all. Any help is greatly appreciated.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/