nleurisource multiple streams

[Thibault Saunier]

Hi,

This is probably related to the fact that the parent bin needs to
handle async state transition in a specific way, I do not remember the
details of why but basically this is what we do in GESTimeline:

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-editing-services/ges/ges-timeline.c#L458

Using NLE from gst-launch is not properly supported as we would need to
do some work for it to properly work (I am not sure what the solution
would be for this one tbh).

Regards,

- Thibault

On Tue, 2022-10-18 at 11:04 -0300, Rodrigo Santos via gstreamer-devel
wrote:
> Hey folks,
> 
> We're working on a video editor software and we've been exploring nle
> plugins. We've tried GES but we ran into some issues and now we're
> trying to understand better how the nle plugins actually work under
> the hood. 
> 
> I'm trying to use nleurisource for media playback. The following
> pipelines work:
> 
> AUDIO ONLY: gst-launch-1.0 nleurisource
> uri=file:///C:/Users/rodsantos/Downloads/video.mp4 caps="audio/x-
> raw(ANY)" ! audioresample ! audioconvert ! autoaudiosink
> 
> VIDEO ONLY: gst-launch-1.0 nleurisource
> uri=file:///C:/Users/rodsantos/Downloads/video.mp4 caps="video/x-
> raw(ANY)" ! videoconvert ! autovideosink
> 
> However, the following pipeline gets stuck in the PREROLLING stage:
> 
> gst-launch-1.0 nleurisource
> uri=file:///C:/Users/rodsantos/Downloads/video.mp4 caps="audio/x-
> raw(ANY); video/x-raw(ANY)" inpoint=5000000000 name=s ! queue !
> audioresample ! audioconvert ! autoaudiosink s. ! queue !
> videoconvert ! autovideosink
> 
> Output:
> 
> Use Windows high-resolution clock, precision: 1 ms 
> Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
> Got context from element 'autovideosink0':
> gst.d3d11.device.handle=context,
> device=(GstD3D11Device)"\(GstD3D11Device\)\ d3d11device2",
> adapter=(uint)0, adapter-luid=(gint64)95670, device-id=(uint)39745,
> vendor-id=(uint)32902, hardware=(boolean)true,
> description=(string)"Intel\(R\)\ UHD\ Graphics";
> Redistribute latency...
> Redistribute latency...
> Redistribute latency...
> Redistribute latency...
> 0:00:00.0 / 0:00:00.0
> 
> Am I doing something wrong for using nleurisource to play both video
> and audio streams? 
> 
> PS: If I just replace nleurisource by uridecodebin, the pipeline
> works as expected.
> 
> Rodrigo Santos
> 
>