capture to file, A/V out of sync

[Nicolas Dufresne]

Le mercredi 14 juillet 2021 à 12:51 +0300, Katerina Voulgary via gstreamer-devel
a écrit :
> Hello, I have a pipeline that does not synchronize audio with video. It 
> seems like it has captured less audio than video.
> 
> VLC plays both streams synchronized, but stalls the video (still frame) 
> at the beginning. MPlayer plays both streams immediately, but audio lags 
> (~1 second).
> 
> I start gstreamer from a bash script for two cameras like this, both 
> files have the same problem,perhaps one more than the other:
> 
>         (gst-launch-1.0 -e pulsesrc device=${audio_array[0]} ! 
> audio/x-raw, rate=8000, channels=2 ! queue !  opusenc name=aenc0 
> audio-type=voice bitrate-type=cbr bandwidth=narrowband ! queue 
> max-size-buffers=0 max-size-bytes=0 max-size-time=5000000000 ! 
> mux0.audio_0 v4l2src device=/dev/${camera_array[0]} ! videoconvert ! 
> queue ! x264enc name=enc0 tune=zerolatency ! queue ! matroskamux 
> name=mux0 ! queue ! filesink location=1$RECORDING_FILEPATH sync=true 
> &>/dev/null & ) &
>          (gst-launch-1.0 -e pulsesrc device=${audio_array[2]} ! 
> audio/x-raw, rate=8000, channels=2 ! queue !  opusenc name=aenc1 
> audio-type=voice bitrate-type=cbr bandwidth=narrowband ! queue 
> max-size-buffers=0 max-size-bytes=0 max-size-time=5000000000 ! 
> mux1.audio_0 v4l2src device=/dev/${camera_array[2]} ! videoconvert ! 
> queue ! x264enc name=enc1 tune=zerolatency ! queue ! matroskamux 
> name=mux1 ! queue ! filesink location=2$RECORDING_FILEPATH sync=true 
> &>/dev/null & ) &
> 
> I was tipped for the parametres at queue to compensate for the x264enc 
> latency. The project is to capture a determined time accurately I am not 
> restricted to container or encoder.
> 
> 1. Could someone give me directions on how to capture synced 
> audio+video? I suspect pulsrc, because I don't know much about it.
> 
> 2. One camera starts later than the other. Can gstreamer add some 
> waiting on the pipeline for both cameras to be ready before it starts 
> capturing or it should be handles externally somehow?

We are basically missing an element, similar in form as multiqueue, but that
would clip the data, so that the chunk that comes out starts aligned. Perhaps it
could be made a multiqueue feature, but as multiqueue is not specific to raw
audio/video, it will be quite challenging.

> 
> 
> GStreamer 1.16.2
> 
> Ubuntu 20.04.2 LTS
> 
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel