compositor + buffering
mariannasb
mariannasb at yahoo.com.br
Mon Jun 13 11:08:00 UTC 2016
Hi, I have a case where I want to use the compositor in order to be able to
mix a live src with a buffered amount of video (which originates from the
same src).
The basic idea is like a instant replay when there is fx a goal in a sports
match.
So I want to have the live data playing all the time, and to keep fx the
last 10 seconds saved in a queue. Then if a goal happens I want to show the
content on the buffer on the same window.
The following pipeline should illustrate a bit the idea:
gst-launch-1.0 videotestsrc is-live=1 ! video/x-raw,width=500,height=500 !
timeoverlay ! tee name=t ! queue ! compositor name=mix sink_1::xpos=100 !
videoconvert ! ximagesink async=0 t. ! queue ! queue max-size-buffers=0
max-size-bytes=0 max-size-time=10000000000 min-threshold-time=10000000000 !
mix.
It almost works as expected.
The problem is that the is a 10 seconds delay before it all starts.
This is what I can see by setting the compositor and aggregators to debug
level 5:
0:00:00.108142457 1404 0x6e0400 DEBUG videoaggregator
gstvideoaggregator.c:1919:gst_videoaggregator_sink_event:<mix:sink_0> Got
segment event on pad mix:sink_0
0:00:00.108244765 1404 0x6e0400 DEBUG videoaggregator
gstvideoaggregator.c:952:gst_videoaggregator_update_qos:<mix> Updating QoS:
proportion 0.500000, diff +0:00:00.000000000, timestamp 99:99:99.999999999
0:00:00.108449201 1404 0x6e0400 DEBUG aggregator
gstaggregator.c:1435:gst_aggregator_query_latency_unlocked:<mix> configured
latency live:true min:10033333333 max:-1
0:00:00.108489586 1404 0x6e0400 DEBUG aggregator
gstaggregator.c:1202:gst_aggregator_default_sink_event:<mix:sink_0> Eating
event: segment event: 0x6c8f10, time 99:99:99.999999999, seq-num 58,
GstEventSegment, segment=(GstSegment)"GstSegment,
flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE, rate=(double)1,
applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, base=(guint64)0,
offset=(guint64)0, start=(guint64)0, stop=(guint64)18446744073709551615,
time=(guint64)0, position=(guint64)0,
duration=(guint64)18446744073709551615;";
0:00:00.108950463 1404 0x6e00f0 DEBUG aggregator
gstaggregator.c:672:gst_aggregator_wait_and_check:<mix> got subclass start
time: 0:00:00.000000000
0:00:00.109017990 1404 0x6e00f0 DEBUG aggregator
gstaggregator.c:690:gst_aggregator_wait_and_check:<mix> possibly waiting for
clock to reach 3:34:48.850798223 (base 3:34:38.817464890 start
0:00:00.000000000 latency 0:00:10.033333333 current 3:34:38.843425373)
0:00:00.109107989 1404 0x6e04a0 DEBUG videoaggregator
gstvideoaggregator.c:1919:gst_videoaggregator_sink_event:<mix:sink_1> Got
segment event on pad mix:sink_1
0:00:00.109138507 1404 0x6e04a0 DEBUG videoaggregator
gstvideoaggregator.c:952:gst_videoaggregator_update_qos:<mix> Updating QoS:
proportion 0.500000, diff +0:00:00.000000000, timestamp 99:99:99.999999999
0:00:00.109181071 1404 0x6e04a0 DEBUG aggregator
gstaggregator.c:1202:gst_aggregator_default_sink_event:<mix:sink_1> Eating
event: segment event: 0x6c8f10, time 99:99:99.999999999, seq-num 58,
GstEventSegment, segment=(GstSegment)"GstSegment,
flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE, rate=(double)1,
applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, base=(guint64)0,
offset=(guint64)0, start=(guint64)0, stop=(guint64)18446744073709551615,
time=(guint64)0, position=(guint64)0,
duration=(guint64)18446744073709551615;";
0:00:00.119823130 1404 0x6e0400 DEBUG aggregator
gstaggregator.c:2148:gst_aggregator_pad_chain_internal:<mix:sink_0> Start
chaining a buffer buffer: 0x7f91d000e060, pts 0:00:00.024814444, dts
99:99:99.999999999, dur 0:00:00.033333333, size 376000, offset 0, offset_end
1, flags 0x40
0:00:00.119966388 1404 0x6e0400 DEBUG aggregator
gstaggregator.c:2251:gst_aggregator_pad_chain_internal:<mix> Selecting start
time 0:00:00.000000000
0:00:00.119998132 1404 0x6e0400 DEBUG aggregator
gstaggregator.c:2263:gst_aggregator_pad_chain_internal:<mix:sink_0> Done
chaining
0:00:00.120258146 1404 0x6e00f0 DEBUG aggregator
gstaggregator.c:707:gst_aggregator_wait_and_check:<mix> clock returned 2
(jitter: -0:00:10.007318606)
0:00:00.120338939 1404 0x6e00f0 DEBUG aggregator
gstaggregator.c:672:gst_aggregator_wait_and_check:<mix> got subclass start
time: 0:00:00.000000000
0:00:00.120375223 1404 0x6e00f0 DEBUG aggregator
gstaggregator.c:690:gst_aggregator_wait_and_check:<mix> possibly waiting for
clock to reach 3:34:48.850798223 (base 3:34:38.817464890 start
0:00:00.000000000 latency 0:00:10.033333333 current 3:34:38.854783033)
0:00:00.144040750 1404 0x6e0400 DEBUG aggregator
gstaggregator.c:2148:gst_aggregator_pad_chain_internal:<mix:sink_0> Start
chaining a buffer buffer: 0x7f91d000e280, pts 0:00:00.058147777, dts
99:99:99.999999999, dur 0:00:00.033333333, size 376000, offset 1, offset_end
2, flags 0x0
0:00:00.144181270 1404 0x6e0400 DEBUG aggregator
gstaggregator.c:2263:gst_aggregator_pad_chain_internal:<mix:sink_0> Done
chaining
0:00:00.144431414 1404 0x6e00f0 DEBUG aggregator
gstaggregator.c:707:gst_aggregator_wait_and_check:<mix> clock returned 2
(jitter: -0:00:09.995947446)
0:00:00.144515261 1404 0x6e00f0 DEBUG aggregator
gstaggregator.c:672:gst_aggregator_wait_and_check:<mix> got subclass start
time: 0:00:00.000000000
0:00:00.144554123 1404 0x6e00f0 DEBUG aggregator
gstaggregator.c:690:gst_aggregator_wait_and_check:<mix> possibly waiting for
clock to reach 3:34:48.850798223 (base 3:34:38.817464890 start
0:00:00.000000000 latency 0:00:10.033333333 current 3:34:38.878961476)
0:00:00.177125923 1404 0x6e0400 DEBUG aggregator
gstaggregator.c:2148:gst_aggregator_pad_chain_internal:<mix:sink_0> Start
chaining a buffer buffer: 0x7f91d000e4a0, pts 0:00:00.091481110, dts
99:99:99.999999999, dur 0:00:00.033333334, size 376000, offset 2, offset_end
3, flags 0x0
0:00:00.177226496 1404 0x6e0400 DEBUG aggregator
gstaggregator.c:2202:gst_aggregator_pad_chain_internal:<mix:sink_0> Waiting
for buffer to be consumed
0:00:10.126676612 1404 0x6e00f0 DEBUG aggregator
gstaggregator.c:707:gst_aggregator_wait_and_check:<mix> clock returned 0
(jitter: -0:00:09.971760916)
0:00:10.127926279 1404 0x6e00f0 DEBUG videoaggregator
gstvideoaggregator.c:718:gst_videoaggregator_update_src_caps:<mix> The
output format will now be : 2 with chroma : jpeg
0:00:10.128376273 1404 0x6e00f0 DEBUG compositor
compositor.c:320:gst_compositor_pad_set_info:<mix:sink_0> This pad will be
converted from 2 to 8
0:00:10.128494739 1404 0x6e00f0 DEBUG compositor
compositor.c:320:gst_compositor_pad_set_info:<mix:sink_1> This pad will be
converted from 2 to 8
0:00:10.128547264 1404 0x6e00f0 INFO videoaggregator
gstvideoaggregator.c:550:gst_videoaggregator_src_setcaps:<mix:src> set src
caps: video/x-raw, width=(int)600, height=(int)500,
framerate=(fraction)30/1, format=(string)BGRx,
pixel-aspect-ratio=(fraction)1/1
0:00:10.128614997 1404 0x6e00f0 INFO aggregator
gstaggregator.c:500:gst_aggregator_push_mandatory_events:<mix> pushing
stream start
0:00:10.128662494 1404 0x6e00f0 FIXME basesink
gstbasesink.c:3125:gst_base_sink_default_event:<ximagesink0> stream-start
event without group-id. Consider implementing group-id handling in the
upstream elements
So from what I can understand the problem is that the compositor queries the
pipeline for the latency and the buffer (queue) is reporting a latency of 10
seconds, which makes it wait 10 seconds to see if it will receive a frame on
sink_1.
Is there any way to avoid this and just make it play without delays?
from gstaggregator.c
gst_aggregator_wait_and_check()
{
...
latency = gst_aggregator_get_latency_unlocked (self);
...
/* If we're not live, or if we use the running time
* of the first buffer as start time, we wait until
* all pads have buffers.
* Otherwise (i.e. if we are live!), we wait on the clock
* and if a pad does not have a buffer in time we ignore
* that pad.
*/
if (!GST_CLOCK_TIME_IS_VALID (latency) || ...){
SRC_WAIT (self);
} else {
...
GST_DEBUG_OBJECT (self, "possibly waiting for clock to reach %" ...)
...
}
...
}
--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/compositor-buffering-tp4678001.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.
More information about the gstreamer-devel
mailing list