Dynamic pipeline (videosink swapping)

[Victor Jaquez]

On Tue, 05 Apr 2022 at 18:37, Rob Kramer via gstreamer-devel wrote:
> On 4/4/22 15:48, Victor Jaquez via gstreamer-devel wrote:
> > 
> > > Thanks again, that seems to select the vampeg2dec element, and seems to work
> > > with my dynamic sink plugger example..
> > > 
> > > In the dot file, there is no mention of VASurface anymore though. Is that
> > > expected, or are things being copied between video and main memory? When
> > > using gst-play-1.0, it does look like the vampeg2dec version uses ~5% less
> > > CPU (~10% vs ~15%).
> > Yes, when the va decoder negotiates with downstream, it tries to check
> > if downstream can handle video meta, if it doesn't (which tee might
> > report drop), va encoder internally downloads and copies the frame onto
> > system memory.
> > 
> > > I've uploaded a dot-file from gst-play, in case you'd like to have a look:
> > > https://pastebin.com/VYPXJP2d
> > Even more, your pipeline does a color conversion in CPU memory.
> 
> No tee was involved here, these were just gst-play tests.
> 
> I thought maybe the va decoder can't negotiate to a VASurface because I had
> disabled the legacy vaapi support and no vaapisink was available. So I
> reconfigured (and threw away build dir to avoid issues) to include gst-vaapi
> as well.
> 
> When I run gst-play (with or without GST_PLUGIN_FEATURE_RANK), this doesn't
> work at all:
> 
>   [laprob] ~/gstreamer> gst-play-1.0 /var/sol/media/video/ATC.mpg
>   Press 'k' to see a list of keyboard shortcuts.
>   Now playing /var/sol/media/video/ATC.mpg
>   Redistribute latency...
> 
>   (gst-play-1.0:193197): GStreamer-Base-CRITICAL **: 18:20:02.880:
> basetransform: second attempt to fixate caps returned invalid (NULL) caps on
> pad vaapipostproc0:sink

I don't know which video sink is picked, but vaapipostproc cannot
convert the frames to what the video sink asks.

And if this is always showed, then you're not setting correctly
GST_PLUGIN_FEATURE_RANK with va elements. vaapipostproc is part of
gstreamer-vaapi and it's used if a gstreamer-vaapi decoder is
autoplugged.

> 
>   -snip 5 more-
> 
>   ERROR Internal data stream error. for file:///var/sol/media/video/ATC.mpg
>   ERROR debug information: ../../gstreamer/subprojects/gst-plugins-bad/gst/mpegdemux/gstmpegdemux.c(3080):
> gst_ps_demux_loop (): /GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMpegPSDemux:mpegpsdemux0:
>   streaming stopped, reason not-negotiated (-4)
>   Reached end of play list.
> 
> When I explicitly request a vaapisink, it works as expected (without
> GST_PLUGIN_FEATURE_RANK), and uses VASurfaces everywhere:
> 
>   GST_DEBUG_DUMP_DOT_DIR=. gst-play-1.0 --videosink=vaapisink
> /var/sol/media/video/ATC.mpg

This works because you set vaapisink, with negotiates correctly with
vaapipostproc.

> 
> The same command with GST_PLUGIN_FEATURE_RANK set works fine too, but
> doesn't use the new vampeg2dec module anywhere. Are these two VA plugins
> able to interoperate, or am I misunderstanding something? I was going to
> upload a level 5 debug output to pastebin, but my browser explodes if I try
> paste it..

They *don't* interoperate. They have different designs. And we don't
have enough bandwith to make them interoperate. We prefer to replace
gstreamer-vaapi as soon as posible.

If you want to avoid confusions, remove gstreamer-vaapi and use
gstreamer 1.20 or main.

> 
> I don't need gst-play btw, but it's nice to test core functionality that
> way, so that I can rule out errors on my side.
> 
> Cheers,
> 
>     Rob
> 
> 
>