Dynamic pipeline (videosink swapping)

[Rob Kramer]

Hi all,

When looking for a method to attach/detach a videosink to a playbin 
pipeline at runtime (in ether PAUSED or PLAYING state), I came accross 
this very helpful blog post:

https://coaxion.net/blog/2014/01/gstreamer-dynamic-pipelines

..in which attaching/detaching videosinks is demonstrated by means of a 
tee element and a permanent fakesink. I've been looking for a way to do 
this for a long time, and have been unsuccessfully messing with just 
swapping a fakesink for a videosink (and back), though I'm not convinced 
that is possible at all (even with blocking pads etc).

The idea is basically to have a video running or paused in the 
background, and sometimes add a videosink to it. Or, to have a few 
pre-rolled headless/fakesink pipelines lying around that can be quickly 
assigned a videosink for instant playback without prerolling pause.

The code in the article can be found at 
https://github.com/sdroege/gst-snippets as dynamic-tee-vsink.c. One 
issue is that the example uses decodebin, whereas my application uses 
playbin and I fear changing that to use decodebin would be very painful.

I modified the example to use playbin instead, and to simplify it for 
now, to only slot in a videosink once, after 2 seconds. My version can 
be found at https://github.com/teegee/gst-snippets/tree/playbin.

The playbin version crashes on many videos, and works OK on others. 
After some comparing of dot-files, it turns out that playing an mpeg1 
movie works, but mpeg2 doesn't. This is possibly because the mpeg2 
pipeline results in a VASurface at the tee sink, and the mpeg1 doesn't. 
I included some logging below, not sure if that is useful.

Does anyone have any idea what could be my issue here?

The mpeg1 version works, but has a significant visual stall when the new 
sink is plugged in, i.e. the first second or so is still. What could 
cause that?

For reference, I used these videos to test:

   mpeg1: 
https://filesamples.com/samples/video/mpeg/sample_960x400_ocean_with_audio.mpeg
   mpeg2: 
https://filesamples.com/samples/video/mpg/sample_960x400_ocean_with_audio.mpg

I'm using gstreamer 1.18 on linux.

Cheers,

      Rob


0:00:02.804661993 59623 0x7fa55c019240 WARN basetransform 
gstbasetransform.c:1370:gst_base_transform_setcaps:<vsbconv> transform 
could not transform video/x-raw(memory:VASurface), format=(string)NV12, 
width=(int)768, height=(int)576, framerate=(fraction)50/1, 
interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, 
multiview-mode=(string)mono, 
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, 
chroma-site=(string)jpeg, colorimetry=(string)bt601 in anything we support

0:00:03.632673414 73705      0x2647980 WARN mpegpsdemux 
gstmpegdemux.c:3066:gst_ps_demux_loop:<mpegpsdemux0> error: Internal 
data stream error.