Can't add video stream to video compositor
Weber, Holger
Holger.Weber at telerob.com
Mon Apr 16 09:33:09 UTC 2018
Hi,
I try to mix different videos into one image. Therefore I use the compositor element.
As source I use three different video files (same format). This is a test setup and the file sources should
be replaced by udpsrc as soon as the test is running.
At the beginning is create the pipeline as shown here:
bin(Filesrc -> tsdemux -> h264parse -> avdec_h264 -> queue) \
-> compositor -> capsfilter -> d3dvideosink
bin(Filesrc -> tsdemux -> h264parse -> avdec_h264 -> queue) /
I add all elements and after that I start the pipline. The output screen shows up and the two videos are mixed into one (ok).
Some time later I want to add a third video stream, while the pipeline is still live. So the pipeline I want to have should look like
below:
bin(Filesrc -> tsdemux -> h264parse -> avdec_h264 -> queue) \
-----> compositor -> capsfilter -> d3dvideosink
bin(Filesrc -> tsdemux -> h264parse -> avdec_h264 -> queue) / /
bin(Filesrc -> tsdemux -> h264parse -> avdec_h264 -> queue) /
I add the third stream as follows:
1) start a timer cb for doing the work
2) cb is called
3) create the new stream bin with all elements
4) add the bin to the pipeline
5) request a new sink pad (sink_2) from the compositor and link it to my bin
6) set GST_STATE_PLAYING for my bin
After adding the third stream I get an internal data stream error and the playback terminates. I see that it has something to do with caps negotiation but I have no idea how to solve that.
I'm also not sure if the order of adding the stream is right.
Below is the output of the video videoaggregator:
0:00:04.340065238 5372 d806120 DEBUG aggregator gstaggregator.c:1370:gst_aggregator_request_new_pad:<mixer> Adding pad sink_2
Linking stream to mixer
0:00:04.340164542 5372 d806120 DEBUG videoaggregator gstvideoaggregator.c:962:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Get caps with filter: (NULL)
0:00:04.340271020 5372 d806120 DEBUG videoaggregator gstvideoaggregator.c:1009:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Returning caps: video/x-raw, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)mixed, format=(string){ Y444, Y42B, YUY2, UYVY, YVYU, I420, YV12, NV12, NV21, Y41B, RGB, BGR, xRGB, xBGR, RGBx, BGRx }
End Of Timeout CB
0:00:04.368477511 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220236, diff -0:00:00.034084034, timestamp 0:00:04.160000000
0:00:04.368641004 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed
0:00:04.368675741 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:04.368707080 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad
0:00:04.368750502 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1
0:00:04.368836969 5372 d999240 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed
0:00:04.400371842 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220695, diff -0:00:00.028304133, timestamp 0:00:04.200000000
0:00:04.400502486 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed
0:00:04.400539111 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:04.400604433 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad
0:00:04.400636149 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1
0:00:04.448112148 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220049, diff -0:00:00.036412067, timestamp 0:00:04.240000000
0:00:04.448252608 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed
0:00:04.448325859 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:04.448353800 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad
0:00:04.448382496 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1
0:00:04.491757956 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220017, diff -0:00:00.028644199, timestamp 0:00:04.280000000
0:00:04.492040387 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed
0:00:04.492075502 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:04.492123455 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad
0:00:04.492152907 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1
0:00:04.522669442 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.221226, diff -0:00:00.024879829, timestamp 0:00:04.320000000
0:00:04.522871826 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed
0:00:04.522912605 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:04.522937903 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad
0:00:04.522967354 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1
0:00:04.559194437 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.221642, diff -0:00:00.034092945, timestamp 0:00:04.360000000
0:00:04.559336785 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed
0:00:04.559370012 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:04.559416832 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad
0:00:04.559484042 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1
0:00:04.611193109 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220725, diff -0:00:00.037583432, timestamp 0:00:04.400000000
0:00:04.611332059 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed
0:00:04.611364531 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:04.611409463 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad
0:00:04.611455906 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1
0:00:04.641116841 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.221058, diff -0:00:00.025611945, timestamp 0:00:04.440000000
0:00:04.641256547 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed
0:00:04.641374352 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:04.641400405 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad
0:00:04.641459308 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1
0:00:04.696486941 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220465, diff -0:00:00.035588154, timestamp 0:00:04.480000000
0:00:04.696580582 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed
0:00:04.696612676 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:04.696635708 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad
0:00:04.696681773 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1
0:00:04.732660029 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.222089, diff -0:00:00.020380036, timestamp 0:00:04.520000000
0:00:04.732872230 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed
0:00:04.732935664 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1
0:00:04.732962095 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad
0:00:04.732990413 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1
Dynamic pad created, linking demuxer/decoder
0:00:04.819389050 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:962:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Get caps with filter: (NULL)
0:00:04.819510254 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1009:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Returning caps: video/x-raw, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)mixed, format=(string){ Y444, Y42B, YUY2, UYVY, YVYU, I420, YV12, NV12, NV21, Y41B, RGB, BGR, xRGB, xBGR, RGBx, BGRx }
0:00:04.879588421 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:962:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Get caps with filter: video/x-raw
0:00:04.879594839 5372 db15400 DEBUG videoaggregator gstvideoaggregator.c:1841:gst_video_aggregator_sink_event:<mixer:sink_2> Got stream-start event on pad mixer:sink_2
0:00:04.879772303 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1009:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Returning caps: video/x-raw, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)mixed, format=(string){ Y444, Y42B, YUY2, UYVY, YVYU, I420, YV12, NV12, NV21, Y41B, RGB, BGR, xRGB, xBGR, RGBx, BGRx }
0:00:04.879798356 5372 db15400 DEBUG aggregator gstaggregator.c:1202:gst_aggregator_default_sink_event:<mixer:sink_2> Eating event: stream-start event: 1cc29c28, time 99:99:99.999999999, seq-num 390, GstEventStreamStart, stream-id=(string)6feac829e54d6d117e303c312fbdde0a9d5adbc93378c4cd4cd4b77a408402a4:1/0000012c, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, stream=(GstStream)"\(GstStream\)\ stream12", group-id=(uint)2;
0:00:04.999976212 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:962:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Get caps with filter: video/x-raw
0:00:05.000110254 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1009:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Returning caps: video/x-raw, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)mixed, format=(string){ Y444, Y42B, YUY2, UYVY, YVYU, I420, YV12, NV12, NV21, Y41B, RGB, BGR, xRGB, xBGR, RGBx, BGRx }
0:00:05.003960455 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:05.004057871 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:05.004126968 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:05.004186626 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:05.004470945 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:05.004551370 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:05.004594792 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:05.004655205 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:05.004737896 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:05.004807748 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:05.004887040 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:05.004957648 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1
Regards,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20180416/7af5cee8/attachment-0001.html>
More information about the gstreamer-devel
mailing list