Dynamic pipeline (videosink swapping)

[Victor Jaquez]

On Thu, 31 Mar 2022 at 16:00, Rob Kramer via gstreamer-devel wrote:
> Hi all,
> 
> When looking for a method to attach/detach a videosink to a playbin pipeline
> at runtime (in ether PAUSED or PLAYING state), I came accross this very
> helpful blog post:
> 
> https://coaxion.net/blog/2014/01/gstreamer-dynamic-pipelines
> 
> ..in which attaching/detaching videosinks is demonstrated by means of a tee
> element and a permanent fakesink. I've been looking for a way to do this for
> a long time, and have been unsuccessfully messing with just swapping a
> fakesink for a videosink (and back), though I'm not convinced that is
> possible at all (even with blocking pads etc).
> 
> The idea is basically to have a video running or paused in the background,
> and sometimes add a videosink to it. Or, to have a few pre-rolled
> headless/fakesink pipelines lying around that can be quickly assigned a
> videosink for instant playback without prerolling pause.
> 
> The code in the article can be found at
> https://github.com/sdroege/gst-snippets as dynamic-tee-vsink.c. One issue is
> that the example uses decodebin, whereas my application uses playbin and I
> fear changing that to use decodebin would be very painful.
> 
> I modified the example to use playbin instead, and to simplify it for now,
> to only slot in a videosink once, after 2 seconds. My version can be found
> at https://github.com/teegee/gst-snippets/tree/playbin.
> 
> The playbin version crashes on many videos, and works OK on others. After
> some comparing of dot-files, it turns out that playing an mpeg1 movie works,
> but mpeg2 doesn't. This is possibly because the mpeg2 pipeline results in a
> VASurface at the tee sink, and the mpeg1 doesn't. I included some logging
> below, not sure if that is useful.
> 
> Does anyone have any idea what could be my issue here?
> 
> The mpeg1 version works, but has a significant visual stall when the new
> sink is plugged in, i.e. the first second or so is still. What could cause
> that?
> 
> For reference, I used these videos to test:
> 
>   mpeg1: https://filesamples.com/samples/video/mpeg/sample_960x400_ocean_with_audio.mpeg
>   mpeg2:
> https://filesamples.com/samples/video/mpg/sample_960x400_ocean_with_audio.mpg
> 
> I'm using gstreamer 1.18 on linux.
> 
> Cheers,
> 
>      Rob
> 
> 
> 0:00:02.804661993 59623 0x7fa55c019240 WARN basetransform
> gstbasetransform.c:1370:gst_base_transform_setcaps:<vsbconv> transform could
> not transform video/x-raw(memory:VASurface), format=(string)NV12,
> width=(int)768, height=(int)576, framerate=(fraction)50/1,
> interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1,
> multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
> chroma-site=(string)jpeg, colorimetry=(string)bt601 in anything we support

You're using gstreamer-vaapi for decoding and VASurface are a scarse
resource which doesn't relate friendly with tee.

Uninstall gstreamer-vaapi and use a sofware-based decoder, or try the
new va plugins in gstreamer 1.20, where the VASurfaces creation is
less restrictive.

vmjl

> 
> 0:00:03.632673414 73705      0x2647980 WARN mpegpsdemux
> gstmpegdemux.c:3066:gst_ps_demux_loop:<mpegpsdemux0> error: Internal data
> stream error.
> 
>