AW: no sync for compositor , glvideomixer

Fri Apr 27 12:48:33 UTC 2018

Hi,

yes the videotestsrc is set to “is-live”.
I also had a look into gstvideoaggregator.c and replaced  the code in the
// } else if (start_time >= output_end_running_time)
block by the code in the if bock before (just to irgnore the running time).

This ugly test leads to two running streams except, that one of them slows down and starts buffering.
As far as I can see the source have different fps an as soon as I add the seconde with 25fps the first slows down from 50 to 25 which
leads to buffering…. What is not what I want.
Is there an easy way to always use the newest buffer from all pads and don’t start buffering or do I need to implement my own mixer?

Von: gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org> Im Auftrag von Nicolas Dufresne
Gesendet: Donnerstag, 26. April 2018 15:28
An: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
Betreff: Re: no sync for compositor , glvideomixer

Le jeu. 26 avr. 2018 05:27, Weber, Holger <Holger.Weber at telerob.com<mailto:Holger.Weber at telerob.com>> a écrit :
Hi,
I try to mix multiple live video streams into one picture with compositor/glvideomixer.
I also add and remove live streams while the pipeline is in PLAYING state.

Adding one stream to the mixer while already playing a videotestsrc works fine. As soon as I add another stream this stream update its image only all 30 to 40 seconds. Removing the first stream causes the second to run.
Does your videotestsrc have is-live=1 set ?
I guess it has something to do with synchronization of the different streams… I get the following output from the aggregator.
It looks like the new stream somehow ended up unaligned. But it's hard to tell, can you provide a self contained example the fails for you ?

0:00:26.670050924  7564    cc42db0 DEBUG             aggregator gstaggregator.c:2495:gst_aggregator_pad_steal_buffer:<mixer:sink_1> Consumed: buffer: 2a83c280, pts 0:00:23.437152317, dts 99:99:99.999999999, dur 0:00:00.020000000, size 1658880, offset none, offset_end none, flags 0x0
0:00:26.670190629  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1564:gst_video_aggregator_aggregate:<mixer> Need more data for decisions
0:00:26.670340906  7564    cc42db0 DEBUG             aggregator gstaggregator.c:675:gst_aggregator_wait_and_check:<mixer> got subclass start time: 0:00:39.366666667
0:00:26.670463620  7564    cc42db0 DEBUG             aggregator gstaggregator.c:690:gst_aggregator_wait_and_check:<mixer> possibly waiting for clock to reach 0:00:40.160052861 (base 0:00:00.000052861 start 0:00:39.366666667 latency 0:00:00.793333333 current 0:00:23.502914735)
0:00:26.670156647  7564    cc42ed0 DEBUG             aggregator gstaggregator.c:2136:gst_aggregator_pad_chain_internal:<mixer:sink_1> Start chaining a buffer buffer: 24b91640, pts 0:00:23.457152317, dts 99:99:99.999999999, dur 0:00:00.020000000, size 1658880, offset none, offset_end none, flags 0x0
0:00:26.670795136  7564    cc42ed0 DEBUG             aggregator gstaggregator.c:2247:gst_aggregator_pad_chain_internal:<mixer:sink_1> Done chaining
0:00:26.670803443  7564    cc42db0 DEBUG             aggregator gstaggregator.c:707:gst_aggregator_wait_and_check:<mixer> clock returned 2 (jitter: -0:00:16.656970102)
0:00:26.670224234  7564    cc42a50 DEBUG             aggregator gstaggregator.c:2247:gst_aggregator_pad_chain_internal:<mixer:sink_0> Done chaining
0:00:26.671047360  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1237:gst_video_aggregator_fill_queues:<mixer:sink_0> Keeping buffer until 0:00:39.400000000
0:00:26.671255030  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1237:gst_video_aggregator_fill_queues:<mixer:sink_2> Keeping buffer until 0:00:50.262005446
0:00:26.671301095  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1226:gst_video_aggregator_fill_queues:<mixer:sink_1> Taking new buffer with start time 0:00:39.357152317
0:00:26.671615997  7564    cc42a50 DEBUG             aggregator gstaggregator.c:2136:gst_aggregator_pad_chain_internal:<mixer:sink_0> Start chaining a buffer buffer: 1e4bfe68, pts 0:00:39.466666666, dts 99:99:99.999999999, dur 0:00:00.033333334, size 307200, offset 1184, offset_end 1185, flags 0x0
0:00:26.671707372  7564    cc42db0 DEBUG             aggregator gstaggregator.c:2495:gst_aggregator_pad_steal_buffer:<mixer:sink_1> Consumed: buffer: 24b91640, pts 0:00:23.457152317, dts 99:99:99.999999999, dur 0:00:00.020000000, size 1658880, offset none, offset_end none, flags 0x0
0:00:26.672007171  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1581:gst_video_aggregator_aggregate:<mixer> Producing buffer for 0:00:39.366666667 to 0:00:39.400000000, running time start 0:00:39.366666667, running time end 0:00:39.400000000
0:00:26.672012457  7564    cc42a50 DEBUG             aggregator gstaggregator.c:2186:gst_aggregator_pad_chain_internal:<mixer:sink_0> Waiting for buffer to be consumed
0:00:26.672633200  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1613:gst_video_aggregator_aggregate:<mixer> Pushing buffer with ts 0:00:39.366666667 and duration 0:00:00.033333333
0:00:26.672772150  7564    cc42db0 DEBUG             aggregator gstaggregator.c:675:gst_aggregator_wait_and_check:<mixer> got subclass start time: 0:00:39.400000000
0:00:26.672854840  7564    cc42db0 DEBUG             aggregator gstaggregator.c:690:gst_aggregator_wait_and_check:<mixer> possibly waiting for clock to reach 0:00:40.193386194 (base 0:00:00.000052861 start 0:00:39.400000000 latency 0:00:00.793333333 current 0:00:23.505308221)
0:00:26.672866923  7564    cc42f90 DEBUG        videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 1.000000, diff +0:00:00.000000000, timestamp 0:00:39.366666667
0:00:26.673118014  7564    cc42f90 DEBUG             aggregator gstaggregator.c:1579:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:26.673215430  7564    cc42f90 DEBUG             aggregator gstaggregator.c:1579:gst_aggregator_event_forward_func:<mixer:sink_2> return of event push is 1
0:00:26.673284150  7564    cc42f90 DEBUG             aggregator gstaggregator.c:1579:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1

For my use case I don’t need any synchronization between  the video sources I want always to mix the newest source picture from each stream into the complete image (reduce latency). I read something about push / pull mode and I wonder if I can change the behaviour of the whole pipeline?

Does anybody has an idea where to start to solve that issue?
There is currently no option to tell the aggregator to ignore time alignment of the input buffers.

_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org<mailto:gstreamer-devel at lists.freedesktop.org>
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20180427/f09cb096/attachment-0001.html>