no sync for compositor , glvideomixer

[Weber, Holger]

Hi,
I try to mix multiple live video streams into one picture with compositor/glvideomixer.
I also add and remove live streams while the pipeline is in PLAYING state.

Adding one stream to the mixer while already playing a videotestsrc works fine. As soon as I add another stream this stream update its image only all 30 to 40 seconds. Removing the first stream causes the second to run.
I guess it has something to do with synchronization of the different streams... I get the following output from the aggregator.

0:00:26.670050924  7564    cc42db0 DEBUG             aggregator gstaggregator.c:2495:gst_aggregator_pad_steal_buffer:<mixer:sink_1> Consumed: buffer: 2a83c280, pts 0:00:23.437152317, dts 99:99:99.999999999, dur 0:00:00.020000000, size 1658880, offset none, offset_end none, flags 0x0
0:00:26.670190629  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1564:gst_video_aggregator_aggregate:<mixer> Need more data for decisions
0:00:26.670340906  7564    cc42db0 DEBUG             aggregator gstaggregator.c:675:gst_aggregator_wait_and_check:<mixer> got subclass start time: 0:00:39.366666667
0:00:26.670463620  7564    cc42db0 DEBUG             aggregator gstaggregator.c:690:gst_aggregator_wait_and_check:<mixer> possibly waiting for clock to reach 0:00:40.160052861 (base 0:00:00.000052861 start 0:00:39.366666667 latency 0:00:00.793333333 current 0:00:23.502914735)
0:00:26.670156647  7564    cc42ed0 DEBUG             aggregator gstaggregator.c:2136:gst_aggregator_pad_chain_internal:<mixer:sink_1> Start chaining a buffer buffer: 24b91640, pts 0:00:23.457152317, dts 99:99:99.999999999, dur 0:00:00.020000000, size 1658880, offset none, offset_end none, flags 0x0
0:00:26.670795136  7564    cc42ed0 DEBUG             aggregator gstaggregator.c:2247:gst_aggregator_pad_chain_internal:<mixer:sink_1> Done chaining
0:00:26.670803443  7564    cc42db0 DEBUG             aggregator gstaggregator.c:707:gst_aggregator_wait_and_check:<mixer> clock returned 2 (jitter: -0:00:16.656970102)
0:00:26.670224234  7564    cc42a50 DEBUG             aggregator gstaggregator.c:2247:gst_aggregator_pad_chain_internal:<mixer:sink_0> Done chaining
0:00:26.671047360  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1237:gst_video_aggregator_fill_queues:<mixer:sink_0> Keeping buffer until 0:00:39.400000000
0:00:26.671255030  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1237:gst_video_aggregator_fill_queues:<mixer:sink_2> Keeping buffer until 0:00:50.262005446
0:00:26.671301095  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1226:gst_video_aggregator_fill_queues:<mixer:sink_1> Taking new buffer with start time 0:00:39.357152317
0:00:26.671615997  7564    cc42a50 DEBUG             aggregator gstaggregator.c:2136:gst_aggregator_pad_chain_internal:<mixer:sink_0> Start chaining a buffer buffer: 1e4bfe68, pts 0:00:39.466666666, dts 99:99:99.999999999, dur 0:00:00.033333334, size 307200, offset 1184, offset_end 1185, flags 0x0
0:00:26.671707372  7564    cc42db0 DEBUG             aggregator gstaggregator.c:2495:gst_aggregator_pad_steal_buffer:<mixer:sink_1> Consumed: buffer: 24b91640, pts 0:00:23.457152317, dts 99:99:99.999999999, dur 0:00:00.020000000, size 1658880, offset none, offset_end none, flags 0x0
0:00:26.672007171  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1581:gst_video_aggregator_aggregate:<mixer> Producing buffer for 0:00:39.366666667 to 0:00:39.400000000, running time start 0:00:39.366666667, running time end 0:00:39.400000000
0:00:26.672012457  7564    cc42a50 DEBUG             aggregator gstaggregator.c:2186:gst_aggregator_pad_chain_internal:<mixer:sink_0> Waiting for buffer to be consumed
0:00:26.672633200  7564    cc42db0 DEBUG        videoaggregator gstvideoaggregator.c:1613:gst_video_aggregator_aggregate:<mixer> Pushing buffer with ts 0:00:39.366666667 and duration 0:00:00.033333333
0:00:26.672772150  7564    cc42db0 DEBUG             aggregator gstaggregator.c:675:gst_aggregator_wait_and_check:<mixer> got subclass start time: 0:00:39.400000000
0:00:26.672854840  7564    cc42db0 DEBUG             aggregator gstaggregator.c:690:gst_aggregator_wait_and_check:<mixer> possibly waiting for clock to reach 0:00:40.193386194 (base 0:00:00.000052861 start 0:00:39.400000000 latency 0:00:00.793333333 current 0:00:23.505308221)
0:00:26.672866923  7564    cc42f90 DEBUG        videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 1.000000, diff +0:00:00.000000000, timestamp 0:00:39.366666667
0:00:26.673118014  7564    cc42f90 DEBUG             aggregator gstaggregator.c:1579:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:26.673215430  7564    cc42f90 DEBUG             aggregator gstaggregator.c:1579:gst_aggregator_event_forward_func:<mixer:sink_2> return of event push is 1
0:00:26.673284150  7564    cc42f90 DEBUG             aggregator gstaggregator.c:1579:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1

For my use case I don't need any synchronization between  the video sources I want always to mix the newest source picture from each stream into the complete image (reduce latency). I read something about push / pull mode and I wonder if I can change the behaviour of the whole pipeline?

Does anybody has an idea where to start to solve that issue?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20180426/18627620/attachment.html>