no sync for compositor , glvideomixer

Nicolas Dufresne nicolas at ndufresne.ca
Thu Apr 26 13:27:59 UTC 2018 

Le jeu. 26 avr. 2018 05:27, Weber, Holger <Holger.Weber at telerob.com> a
écrit :

> Hi,
>
> I try to mix multiple live video streams into one picture with
> compositor/glvideomixer.
>
> I also add and remove live streams while the pipeline is in PLAYING state.
>
>
>
> Adding one stream to the mixer while already playing a videotestsrc works
> fine. As soon as I add another stream this stream update its image only all
> 30 to 40 seconds. Removing the first stream causes the second to run.
>
Does your videotestsrc have is-live=1 set ?

> I guess it has something to do with synchronization of the different
> streams… I get the following output from the aggregator.
>
It looks like the new stream somehow ended up unaligned. But it's hard to
tell, can you provide a self contained example the fails for you ?

>
>
> 0:00:26.670050924  7564    cc42db0 DEBUG             aggregator
> gstaggregator.c:2495:gst_aggregator_pad_steal_buffer:<mixer:sink_1>
> Consumed: buffer: 2a83c280, pts 0:00:23.437152317, dts 99:99:99.999999999,
> dur 0:00:00.020000000, size 1658880, offset none, offset_end none, flags 0x0
>
> 0:00:26.670190629  7564    cc42db0 DEBUG        videoaggregator
> gstvideoaggregator.c:1564:gst_video_aggregator_aggregate:<mixer> Need more
> data for decisions
>
> 0:00:26.670340906  7564    cc42db0 DEBUG             aggregator
> gstaggregator.c:675:gst_aggregator_wait_and_check:<mixer> got subclass
> start time: 0:00:39.366666667
>
> 0:00:26.670463620  7564    cc42db0 DEBUG             aggregator
> gstaggregator.c:690:gst_aggregator_wait_and_check:<mixer> possibly waiting
> for clock to reach 0:00:40.160052861 (base 0:00:00.000052861 start
> 0:00:39.366666667 latency 0:00:00.793333333 current 0:00:23.502914735)
>
> 0:00:26.670156647  7564    cc42ed0 DEBUG             aggregator
> gstaggregator.c:2136:gst_aggregator_pad_chain_internal:<mixer:sink_1> Start
> chaining a buffer buffer: 24b91640, pts 0:00:23.457152317, dts
> 99:99:99.999999999, dur 0:00:00.020000000, size 1658880, offset none,
> offset_end none, flags 0x0
>
> 0:00:26.670795136  7564    cc42ed0 DEBUG             aggregator
> gstaggregator.c:2247:gst_aggregator_pad_chain_internal:<mixer:sink_1> Done
> chaining
>
> 0:00:26.670803443  7564    cc42db0 DEBUG             aggregator
> gstaggregator.c:707:gst_aggregator_wait_and_check:<mixer> clock returned 2
> (jitter: -0:00:16.656970102)
>
> 0:00:26.670224234  7564    cc42a50 DEBUG             aggregator
> gstaggregator.c:2247:gst_aggregator_pad_chain_internal:<mixer:sink_0> Done
> chaining
>
> 0:00:26.671047360  7564    cc42db0 DEBUG        videoaggregator
> gstvideoaggregator.c:1237:gst_video_aggregator_fill_queues:<mixer:sink_0>
> Keeping buffer until 0:00:39.400000000
>
> 0:00:26.671255030  7564    cc42db0 DEBUG        videoaggregator
> gstvideoaggregator.c:1237:gst_video_aggregator_fill_queues:<mixer:sink_2>
> Keeping buffer until 0:00:50.262005446
>
> 0:00:26.671301095  7564    cc42db0 DEBUG        videoaggregator
> gstvideoaggregator.c:1226:gst_video_aggregator_fill_queues:<mixer:sink_1>
> Taking new buffer with start time 0:00:39.357152317
>
> 0:00:26.671615997  7564    cc42a50 DEBUG             aggregator
> gstaggregator.c:2136:gst_aggregator_pad_chain_internal:<mixer:sink_0> Start
> chaining a buffer buffer: 1e4bfe68, pts 0:00:39.466666666, dts
> 99:99:99.999999999, dur 0:00:00.033333334, size 307200, offset 1184,
> offset_end 1185, flags 0x0
>
> 0:00:26.671707372  7564    cc42db0 DEBUG             aggregator
> gstaggregator.c:2495:gst_aggregator_pad_steal_buffer:<mixer:sink_1>
> Consumed: buffer: 24b91640, pts 0:00:23.457152317, dts 99:99:99.999999999,
> dur 0:00:00.020000000, size 1658880, offset none, offset_end none, flags 0x0
>
> 0:00:26.672007171  7564    cc42db0 DEBUG        videoaggregator
> gstvideoaggregator.c:1581:gst_video_aggregator_aggregate:<mixer> Producing
> buffer for 0:00:39.366666667 to 0:00:39.400000000, running time start
> 0:00:39.366666667, running time end 0:00:39.400000000
>
> 0:00:26.672012457  7564    cc42a50 DEBUG             aggregator
> gstaggregator.c:2186:gst_aggregator_pad_chain_internal:<mixer:sink_0>
> Waiting for buffer to be consumed
>
> 0:00:26.672633200  7564    cc42db0 DEBUG        videoaggregator
> gstvideoaggregator.c:1613:gst_video_aggregator_aggregate:<mixer> Pushing
> buffer with ts 0:00:39.366666667 and duration 0:00:00.033333333
>
> 0:00:26.672772150  7564    cc42db0 DEBUG             aggregator
> gstaggregator.c:675:gst_aggregator_wait_and_check:<mixer> got subclass
> start time: 0:00:39.400000000
>
> 0:00:26.672854840  7564    cc42db0 DEBUG             aggregator
> gstaggregator.c:690:gst_aggregator_wait_and_check:<mixer> possibly waiting
> for clock to reach 0:00:40.193386194 (base 0:00:00.000052861 start
> 0:00:39.400000000 latency 0:00:00.793333333 current 0:00:23.505308221)
>
> 0:00:26.672866923  7564    cc42f90 DEBUG        videoaggregator
> gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating
> QoS: proportion 1.000000, diff +0:00:00.000000000, timestamp
> 0:00:39.366666667
>
> 0:00:26.673118014  7564    cc42f90 DEBUG             aggregator
> gstaggregator.c:1579:gst_aggregator_event_forward_func:<mixer:sink_0>
> return of event push is 1
>
> 0:00:26.673215430  7564    cc42f90 DEBUG             aggregator
> gstaggregator.c:1579:gst_aggregator_event_forward_func:<mixer:sink_2>
> return of event push is 1
>
> 0:00:26.673284150  7564    cc42f90 DEBUG             aggregator
> gstaggregator.c:1579:gst_aggregator_event_forward_func:<mixer:sink_1>
> return of event push is 1
>
>
>
> For my use case I don’t need any synchronization between  the video
> sources I want always to mix the newest source picture from each stream
> into the complete image (reduce latency). I read something about push /
> pull mode and I wonder if I can change the behaviour of the whole pipeline?
>
>
>
> Does anybody has an idea where to start to solve that issue?
>
There is currently no option to tell the aggregator to ignore time
alignment of the input buffers.


_______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20180426/8e0ddc8c/attachment.html>

More information about the gstreamer-devel mailing list