GStreamer multi pipeline time sync

[Peter Biro]

Hi

I opened a separate thread about queueing the muxed data, so here the the
question that remained is what am I doing wrong when I try to block and
unblock the two separate queues before the mux.

Is there any material I should read through in more detail? Or do you have
any advice where I should look in the logs since right now i'm a bit lost.

I tried to execute the blockin and blockings of the pads serially(block the
second queue from the first's callback) and in parallel but that did not
help. Or what i'm trying to do is conceptually wrong?

Thanks!

Peter

On Mon, Dec 7, 2020 at 12:00 AM Peter Biro <pettair at gmail.com> wrote:

> Hi,
>
>
> I refactored my project based on Tim's examples, which works perfectly
> when the pipeline contains only a video source, although I found one
> strange thing in the behaviour:
> the blocking of a pad work only in cases when there is a "tee" between the
> source and the blocked element, otherwise it will block the other parts of
> the pipeline (so I added on extra(unnecessary?) tee for the audio). Is this
> expected or I do something not properly?
>
> But when I add an audio source things are getting a bit more complicated.
>
> So I added the audio and moved the cacheing/buffering queue  after the
> muxer () . But this way it generates invalid file on the output (VLC show
> this kind of issues: "ps warning: this does not look like an MPEG PS
> stream, continuing anyway; ps warning: garbage at input from 509, trying to
> resync...")
>
> nvarguscamerasrc sensor-id=0 sensor-mode=0
> ! video/x-raw(memory:NVMM), width=(int)1980, height=(int)1080,
> format=(string)NV12, framerate=(fraction)20/1
> ! nvvidconv ! textoverlay name=text_overlay ! video/x-raw,format=I420 !
> nvvidconv ! nvv4l2vp8enc ! tee name=video_stream_spilt
> audiotestsrc ! vorbisenc ! queue ! file_sink_video_mux.
> video_stream_spilt. ! queue ! webmmux name=file_sink_video_mux ! tee
> name=muxed_video_stream_spilt
> muxed_video_stream_spilt. ! queue name=file_sink_queue ! filesink
> name=file_sink location=/tmp/video_out.webm
> muxed_video_stream_spilt. ! fakesink
> video_stream_spilt. ! rtpvp8pay mtu=1400 ! appsink name=app_sink
>
> I tried to google around, but I not found any similar examples so I guess
> I cannot "buffer" muxed packets with a queue. Is this approach
> fundamentally wrong?
>
> After this I tried to add queues before the mux, but this way the pipeline
> stops when Im unblocking the pads.
>
> nvarguscamerasrc sensor-id=0 sensor-mode=0
> ! video/x-raw(memory:NVMM), width=(int)1980, height=(int)1080,
> format=(string)NV12, framerate=(fraction)20/1
> ! nvvidconv ! textoverlay name=text_overlay ! video/x-raw,format=I420 !
> nvvidconv ! nvv4l2vp8enc ! tee name=video_stream_spilt
> audiotestsrc ! vorbisenc ! tee name=audio_stream_spilt
> audio_stream_spilt. ! queue name=file_sink_audio_queue !
> file_sink_video_mux.
> audio_stream_spilt. ! fakesink
> video_stream_spilt. ! queue name=file_sink_video_queue ! webmmux
> name=file_sink_video_mux ! filesink name=file_sink
> location=/tmp/video_out.webm
> video_stream_spilt. ! rtpvp8pay mtu=1400 ! appsink name=app_sink
>
> I will attach the debug output, the corresponding part comes after the "startRecording:
> dcd - startRecording" log message. Im sure that im doing something wrong,
> since I think this should work.
>
> Do you have any idea what am I doing wrong?
>
> Thank you!
>
>
> On 2020. Dec 2., at 21:27, Peter Biro <pettair at gmail.com> wrote:
>
> Hi Tim.
>
> Thanks for the examples!
>
> I think I tried something similar before but as I remember when I set the
> state on the filesink (to be able to change the location) to GST_STATE_NULL
> it was propagated trough the pipeline and it caused glitches in the
> streaming.
>
> As I see you do kind of the same here with the 'autovideosink' and I guess
> you are not experience such thing since you block the pad on 'vrecq' to
> block?
>
> Bests,
> Peter
>
> On 2020. Dec 2., at 0:30, Tim Müller <tim at centricular.com> wrote:
>
> Hi Peter,
>
> I have an example for save-to-file-with-backlog here:
>
> https://people.freedesktop.org/~tpm/code/test-backlog-recording-h264.c
>
> or (rtp variant):
>
> https://people.freedesktop.org/~tpm/code/test-backlog-recording-h264-rtp.c
>
> for what it's worth.
>
> And audiomixer element can produce silence samples if it operates in
> live mode, which will happen if the upstream source is a live/capture
> source or you force it into live mode with a dummy audiotestsrc is-
> live=true branch.
>
> Alternatively  .. ! interaudiosink   interaudiosrc ! ...
>
> Cheers
> Tim
>
> --
> Tim Müller, Centricular Ltd - http://www.centricular.com
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20201210/0520042c/attachment.htm>