<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="DE" link="#0563C1" vlink="#954F72"> <div class="WordSection1"> Hi,<o:p></o:p> <o:p> </o:p> I try to mix different videos into one image. Therefore I use the compositor element.<o:p></o:p> As source I use three different video files (same format). This is a test setup and the file sources should<o:p></o:p> be replaced by udpsrc as soon as the test is running.<o:p></o:p> <o:p> </o:p> At the beginning is create the pipeline as shown here:<o:p></o:p> <o:p> </o:p> bin(Filesrc -> tsdemux -> h264parse -> avdec_h264 -> queue) \<o:p></o:p> -> compositor -> capsfilter -> d3dvideosink<o:p></o:p> bin(Filesrc -> tsdemux -> h264parse -> avdec_h264 -> queue) /<o:p></o:p> <o:p> </o:p> I add all elements and after that I start the pipline. The output screen shows up and the two videos are mixed into one (ok).<o:p></o:p> <o:p> </o:p> Some time later I want to add a third video stream, while the pipeline is still live. So the pipeline I want to have should look like<o:p></o:p> below:<o:p></o:p> <o:p> </o:p> bin(Filesrc -> tsdemux -> h264parse -> avdec_h264 -> queue) \<o:p></o:p> -----> compositor -> capsfilter -> d3dvideosink<o:p></o:p> bin(Filesrc -> tsdemux -> h264parse -> avdec_h264 -> queue) / /<o:p></o:p> bin(Filesrc -> tsdemux -> h264parse -> avdec_h264 -> queue) /<o:p></o:p> <o:p> </o:p> I add the third stream as follows:<o:p></o:p> 1) start a timer cb for doing the work<o:p></o:p> 2) cb is called <o:p></o:p> 3) create the new stream bin with all elements <o:p></o:p> 4) add the bin to the pipeline<o:p></o:p> 5) request a new sink pad (sink_2) from the compositor and link it to my bin<o:p></o:p> 6) set GST_STATE_PLAYING for my bin<o:p></o:p> <o:p> </o:p> After adding the third stream I get an internal data stream error and the playback terminates. I see that it has something to do with caps negotiation but I have no idea how to solve that.<o:p></o:p> I'm also not sure if the order of adding the stream is right.<o:p></o:p> <o:p> </o:p> Below is the output of the video videoaggregator:<o:p></o:p> <o:p> </o:p> <o:p> </o:p> <o:p> </o:p> 0:00:04.340065238 5372 d806120 DEBUG aggregator gstaggregator.c:1370:gst_aggregator_request_new_pad:<mixer> Adding pad sink_2<o:p></o:p> Linking stream to mixer<o:p></o:p> 0:00:04.340164542 5372 d806120 DEBUG videoaggregator gstvideoaggregator.c:962:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Get caps with filter: (NULL)<o:p></o:p> 0:00:04.340271020 5372 d806120 DEBUG videoaggregator gstvideoaggregator.c:1009:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Returning caps: video/x-raw, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)mixed, format=(string){ Y444, Y42B, YUY2, UYVY, YVYU, I420, YV12, NV12, NV21, Y41B, RGB, BGR, xRGB, xBGR, RGBx, BGRx }<o:p></o:p> End Of Timeout CB<o:p></o:p> 0:00:04.368477511 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220236, diff -0:00:00.034084034, timestamp 0:00:04.160000000<o:p></o:p> 0:00:04.368641004 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed<o:p></o:p> 0:00:04.368675741 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1<o:p></o:p> 0:00:04.368707080 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad<o:p></o:p> 0:00:04.368750502 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1<o:p></o:p> 0:00:04.368836969 5372 d999240 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed<o:p></o:p> 0:00:04.400371842 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220695, diff -0:00:00.028304133, timestamp 0:00:04.200000000<o:p></o:p> 0:00:04.400502486 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed<o:p></o:p> 0:00:04.400539111 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1<o:p></o:p> 0:00:04.400604433 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad<o:p></o:p> 0:00:04.400636149 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1<o:p></o:p> 0:00:04.448112148 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220049, diff -0:00:00.036412067, timestamp 0:00:04.240000000<o:p></o:p> 0:00:04.448252608 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed<o:p></o:p> 0:00:04.448325859 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1<o:p></o:p> 0:00:04.448353800 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad<o:p></o:p> 0:00:04.448382496 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1<o:p></o:p> 0:00:04.491757956 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220017, diff -0:00:00.028644199, timestamp 0:00:04.280000000<o:p></o:p> 0:00:04.492040387 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed<o:p></o:p> 0:00:04.492075502 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1<o:p></o:p> 0:00:04.492123455 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad<o:p></o:p> 0:00:04.492152907 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1<o:p></o:p> 0:00:04.522669442 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.221226, diff -0:00:00.024879829, timestamp 0:00:04.320000000<o:p></o:p> 0:00:04.522871826 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed<o:p></o:p> 0:00:04.522912605 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1<o:p></o:p> 0:00:04.522937903 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad<o:p></o:p> 0:00:04.522967354 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1<o:p></o:p> 0:00:04.559194437 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.221642, diff -0:00:00.034092945, timestamp 0:00:04.360000000<o:p></o:p> 0:00:04.559336785 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed<o:p></o:p> 0:00:04.559370012 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1<o:p></o:p> 0:00:04.559416832 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad<o:p></o:p> 0:00:04.559484042 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1<o:p></o:p> 0:00:04.611193109 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220725, diff -0:00:00.037583432, timestamp 0:00:04.400000000<o:p></o:p> 0:00:04.611332059 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed<o:p></o:p> 0:00:04.611364531 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1<o:p></o:p> 0:00:04.611409463 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad<o:p></o:p> 0:00:04.611455906 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1<o:p></o:p> 0:00:04.641116841 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.221058, diff -0:00:00.025611945, timestamp 0:00:04.440000000<o:p></o:p> 0:00:04.641256547 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed<o:p></o:p> 0:00:04.641374352 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1<o:p></o:p> 0:00:04.641400405 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad<o:p></o:p> 0:00:04.641459308 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1<o:p></o:p> 0:00:04.696486941 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.220465, diff -0:00:00.035588154, timestamp 0:00:04.480000000<o:p></o:p> 0:00:04.696580582 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed<o:p></o:p> 0:00:04.696612676 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1<o:p></o:p> 0:00:04.696635708 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad<o:p></o:p> 0:00:04.696681773 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1<o:p></o:p> 0:00:04.732660029 5372 d999270 DEBUG videoaggregator gstvideoaggregator.c:1023:gst_video_aggregator_update_qos:<mixer> Updating QoS: proportion 0.222089, diff -0:00:00.020380036, timestamp 0:00:04.520000000<o:p></o:p> 0:00:04.732872230 5372 d999270 WARN aggregator gstaggregator.c:1393:gst_aggregator_query_latency_unlocked:<mixer> Latency query failed<o:p></o:p> 0:00:04.732935664 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_0> return of event push is 1<o:p></o:p> 0:00:04.732962095 5372 d999270 DEBUG aggregator gstaggregator.c:1571:gst_aggregator_event_forward_func:<mixer:sink_2> not sending event to inactive pad<o:p></o:p> 0:00:04.732990413 5372 d999270 DEBUG aggregator gstaggregator.c:1575:gst_aggregator_event_forward_func:<mixer:sink_1> return of event push is 1<o:p></o:p> Dynamic pad created, linking demuxer/decoder<o:p></o:p> 0:00:04.819389050 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:962:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Get caps with filter: (NULL)<o:p></o:p> 0:00:04.819510254 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1009:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Returning caps: video/x-raw, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)mixed, format=(string){ Y444, Y42B, YUY2, UYVY, YVYU, I420, YV12, NV12, NV21, Y41B, RGB, BGR, xRGB, xBGR, RGBx, BGRx }<o:p></o:p> 0:00:04.879588421 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:962:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Get caps with filter: video/x-raw<o:p></o:p> 0:00:04.879594839 5372 db15400 DEBUG videoaggregator gstvideoaggregator.c:1841:gst_video_aggregator_sink_event:<mixer:sink_2> Got stream-start event on pad mixer:sink_2<o:p></o:p> 0:00:04.879772303 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1009:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Returning caps: video/x-raw, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)mixed, format=(string){ Y444, Y42B, YUY2, UYVY, YVYU, I420, YV12, NV12, NV21, Y41B, RGB, BGR, xRGB, xBGR, RGBx, BGRx }<o:p></o:p> 0:00:04.879798356 5372 db15400 DEBUG aggregator gstaggregator.c:1202:gst_aggregator_default_sink_event:<mixer:sink_2> Eating event: stream-start event: 1cc29c28, time 99:99:99.999999999, seq-num 390, GstEventStreamStart, stream-id=(string)6feac829e54d6d117e303c312fbdde0a9d5adbc93378c4cd4cd4b77a408402a4:1/0000012c, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, stream=(GstStream)"\(GstStream\)\ stream12", group-id=(uint)2;<o:p></o:p> 0:00:04.999976212 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:962:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Get caps with filter: video/x-raw<o:p></o:p> 0:00:05.000110254 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1009:gst_video_aggregator_pad_sink_getcaps:<mixer:sink_2> Returning caps: video/x-raw, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)mixed, format=(string){ Y444, Y42B, YUY2, UYVY, YVYU, I420, YV12, NV12, NV21, Y41B, RGB, BGR, xRGB, xBGR, RGBx, BGRx }<o:p></o:p> 0:00:05.003960455 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> 0:00:05.004057871 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> 0:00:05.004126968 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> 0:00:05.004186626 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> 0:00:05.004470945 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> 0:00:05.004551370 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> 0:00:05.004594792 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> 0:00:05.004655205 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> 0:00:05.004737896 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> 0:00:05.004807748 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> 0:00:05.004887040 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:1990:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> 0:00:05.004957648 5372 db155b0 DEBUG videoaggregator gstvideoaggregator.c:2020:gst_video_aggregator_pad_sink_acceptcaps:<mixer:sink_2> not accepted caps video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)12/11, chroma-site=(string)mpeg2, colorimetry=(string)bt601, framerate=(fraction)25/1<o:p></o:p> <o:p> </o:p> Regards,<o:p></o:p> </div> </body> </html>