NLE composition inconsistent duration

[Lucas McGartland]

Thibault,



Managed to get the pipeline properly writing to a file by setting the proper caps for the framerate.



However, we're unable to output to autovideosink due to a QoS event.

videodecoder gstvideodecoder.c:3675:gst_video_decoder_clip_and_push_buf:<avdec_h264-1> Dropping frame due to QoS. start:0:00:00.000000000 deadline:0:00:00.000000000 earliest_time:0:00:03.034568335

All the frames get dropped because they are arriving "late"



How would we go about updating the latency for this?



And regarding GES, we have some specific requirements that need finer grained control of the timeline state than what GES allows.


Thanks,

Luke






---- On Tue, 11 Oct 2022 09:58:48 -0700 Thibault Saunier <thibault at saunier.eu> wrote ---



Hi Rodrigo, 
 
 
That sounds weird I just tested your program and it just works here, 
ooc, what version of Gst are you using? 
 
By the way, you should also commit the composition, emitting the 
'commit' signal (but for this simple case it will be done 
automatically). 
 
Also, have you checked the GStreamer Editing Services? It is simpler to 
use and mighthelp with your use case. 
 
Regards, 
 
- Thibault 
 
On Tue, 2022-10-11 at 11:53 -0300, Rodrigo Santos via gstreamer-devel 
wrote: 
> Hi, 
> 
> I have the following code in Rust that creates a nlecomposition and 
> adds to it two nleurisource elements. The duration of both sources is 
> set to 10s, and the start of the second source is set to 10s. The 
> output is being stored in a file. 
> 
> My expectation is that the video file would have a total of 20s of 
> duration, but its actual duration is 33s: the first video has a 
> duration of 22s while the second video has 11s of duration. Is this a 
> bug or am I missing something? 
> 
> The weird thing is if I don't change the uri when I create the second 
> source (both sources having the same value for the uri property) the 
> resulting video file will have 20s of duration as expected (each 
> video plays for 10s). 
> 
> fn add_to_bin(bin: &Element, element: &Element) { 
>     let bin = bin.dynamic_cast_ref::<Bin>().unwrap(); 
>     bin.add_many(&[element]).unwrap(); 
> } 
> 
> fn create_nle_src (uri: &str, start: u64, inpoint: u64, duration: 
> i64, priority: u32) -> Element { 
>     let nlesource = ElementFactory::make("nleurisource", 
> None).unwrap(); 
> 
>     nlesource.set_property("uri", uri); 
>     nlesource.set_property("start", start); 
>     nlesource.set_property("inpoint", inpoint); 
>     nlesource.set_property("duration", duration); 
>     nlesource.set_property("priority", priority); 
> 
>     nlesource 
> } 
> 
> fn main() { 
>     <....> 
>     let comp = ElementFactory::make("nlecomposition", Some("test- 
> composition")).unwrap(); 
>     let caps = "video/x-raw(ANY)"; 
>     let caps = Caps::from_str(caps).expect("Could not create caps 
> from str"); 
>     comp.set_property("caps", caps); 
> 
>     let uri = "file:///C:/Users/rodsantos/Downloads/video.mp4"; //big 
> buck bunny video 
>     let inpoint = ClockTime::from_seconds(0).nseconds(); 
>     let duration: i64 = 
> ClockTime::from_seconds(10).nseconds().try_into().expect("Cast 
> failed"); 
>     let mut start: u64 = 0; 
>     let nlesource= create_nle_src(uri, start, inpoint, duration, 1); 
> 
>     add_to_bin(&comp, &nlesource); 
> 
>     start += duration as u64; 
> 
>     let uri = "file:///C:/Users/rodsantos/Downloads/sintel_trailer- 
> 480p.webm"; //sintel trailer video 
>     let duration: i64 = 
> ClockTime::from_seconds(10).nseconds().try_into().expect("Cast 
> failed"); 
>     let nlesource = create_nle_src(uri, start, inpoint, duration, 1); 
>     add_to_bin(&comp, &nlesource); 
> 
>     comp.emit_by_name::<bool>("commit", &[&true]); 
> 
>     let videoconvert = ElementFactory::make("videoconvert", 
> None).unwrap(); 
>     let compositor = ElementFactory::make("compositor", 
> None).unwrap(); 
>     let encoder = ElementFactory::make("x264enc", None).unwrap(); 
>     let mux = ElementFactory::make("avimux", None).unwrap(); 
>     let filesink = ElementFactory::make("filesink", None).unwrap(); 
> 
>     let pipeline = Pipeline::new(None); 
>     pipeline.add_many(&[ 
>         &comp, 
>         &videoconvert, 
>         &compositor, 
>         &encoder, 
>         &mux, 
>         &filesink, 
>     ]).unwrap(); 
> 
>     Element::link_many(&[ 
>         &comp, 
>         &videoconvert, 
>         &compositor, 
>         &encoder, 
>         &mux, 
>         &filesink 
>     ]).unwrap(); 
> 
>     filesink.set_property("location", "test.avi"); 
> 
>     pipeline.set_state(State::Playing).expect("Could not set pipeline 
> to Playing state"); 
>     
>    <....> 
> } 
> 
> 
> Rodrigo Santos 
> 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20221012/0f38cb0c/attachment.htm>