GStreamer multi pipeline time sync

[Peter Biro]

Hi,


I refactored my project based on Tim's examples, which works perfectly when the pipeline contains only a video source, although I found one strange thing in the behaviour: 
the blocking of a pad work only in cases when there is a "tee" between the source and the blocked element, otherwise it will block the other parts of the pipeline (so I added on extra(unnecessary?) tee for the audio). Is this expected or I do something not properly?  

But when I add an audio source things are getting a bit more complicated.

So I added the audio and moved the cacheing/buffering queue  after the muxer () . But this way it generates invalid file on the output (VLC show this kind of issues: "ps warning: this does not look like an MPEG PS stream, continuing anyway; ps warning: garbage at input from 509, trying to resync...")

nvarguscamerasrc sensor-id=0 sensor-mode=0
      ! video/x-raw(memory:NVMM), width=(int)1980, height=(int)1080, format=(string)NV12, framerate=(fraction)20/1 
      ! nvvidconv ! textoverlay name=text_overlay ! video/x-raw,format=I420 ! nvvidconv ! nvv4l2vp8enc ! tee name=video_stream_spilt
      audiotestsrc ! vorbisenc ! queue ! file_sink_video_mux.
      video_stream_spilt. ! queue ! webmmux name=file_sink_video_mux ! tee name=muxed_video_stream_spilt
      muxed_video_stream_spilt. ! queue name=file_sink_queue ! filesink name=file_sink location=/tmp/video_out.webm
      muxed_video_stream_spilt. ! fakesink
      video_stream_spilt. ! rtpvp8pay mtu=1400 ! appsink name=app_sink

I tried to google around, but I not found any similar examples so I guess I cannot "buffer" muxed packets with a queue. Is this approach fundamentally wrong?

After this I tried to add queues before the mux, but this way the pipeline stops when Im unblocking the pads. 

nvarguscamerasrc sensor-id=0 sensor-mode=0
      ! video/x-raw(memory:NVMM), width=(int)1980, height=(int)1080, format=(string)NV12, framerate=(fraction)20/1 
      ! nvvidconv ! textoverlay name=text_overlay ! video/x-raw,format=I420 ! nvvidconv ! nvv4l2vp8enc ! tee name=video_stream_spilt 
      audiotestsrc ! vorbisenc ! tee name=audio_stream_spilt
      audio_stream_spilt. ! queue name=file_sink_audio_queue ! file_sink_video_mux.
      audio_stream_spilt. ! fakesink
      video_stream_spilt. ! queue name=file_sink_video_queue ! webmmux name=file_sink_video_mux ! filesink name=file_sink location=/tmp/video_out.webm
      video_stream_spilt. ! rtpvp8pay mtu=1400 ! appsink name=app_sink

I will attach the debug output, the corresponding part comes after the "startRecording: dcd - startRecording" log message. Im sure that im doing something wrong, since I think this should work. 

Do you have any idea what am I doing wrong?

Thank you!



> On 2020. Dec 2., at 21:27, Peter Biro <pettair at gmail.com> wrote:
> 
> Hi Tim.
> 
> Thanks for the examples!
> 
> I think I tried something similar before but as I remember when I set the state on the filesink (to be able to change the location) to GST_STATE_NULL it was propagated trough the pipeline and it caused glitches in the streaming. 
> 
> As I see you do kind of the same here with the 'autovideosink' and I guess you are not experience such thing since you block the pad on 'vrecq' to block? 
> 
> Bests,
> Peter
> 
>> On 2020. Dec 2., at 0:30, Tim Müller <tim at centricular.com> wrote:
>> 
>> Hi Peter,
>> 
>> I have an example for save-to-file-with-backlog here:
>> 
>> https://people.freedesktop.org/~tpm/code/test-backlog-recording-h264.c
>> 
>> or (rtp variant):
>> 
>> https://people.freedesktop.org/~tpm/code/test-backlog-recording-h264-rtp.c
>> 
>> for what it's worth.
>> 
>> And audiomixer element can produce silence samples if it operates in
>> live mode, which will happen if the upstream source is a live/capture
>> source or you force it into live mode with a dummy audiotestsrc is-
>> live=true branch.
>> 
>> Alternatively  .. ! interaudiosink   interaudiosrc ! ...
>> 
>> Cheers
>> Tim
>> 
>> -- 
>> Tim Müller, Centricular Ltd - http://www.centricular.com
>> 
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20201207/09f21569/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gst.log
Type: application/octet-stream
Size: 378240 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20201207/09f21569/attachment-0001.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20201207/09f21569/attachment-0003.htm>