How to correctly timestamp buffers in an appsrc element

[J. Krieg]

Hello Nicolas,

Thanks for your response.

I did so as you suggested and also a step back focusing only on the
video part to get a smooth playback.
But I still get a stuttering video output.

I see the following log lines in the debug output:
0:00:03.200317080  2248   0xdf1630 DEBUG           videodecoder
gstvideodecoder.c:1575:gst_video_decoder_src_event:<vdecoder> received
event 48641, qos
0:00:03.201372809  2248   0xdf1630 DEBUG           videodecoder
gstvideodecoder.c:1476:gst_video_decoder_src_event_default:<vdecoder>
received event 48641, qos
0:00:03.202372548  2248   0xdf1630 DEBUG           videodecoder
gstvideodecoder.c:1547:gst_video_decoder_src_event_default:<vdecoder>
got QoS 0:00:01.740711589, +0:00:00.068395216, 0.77583
0:00:03.221023165  2248   0xdf1690 DEBUG           videodecoder
gstvideodecoder.c:3408:gst_video_decoder_decode_frame:<vdecoder>
decoder frame list getting long: 12 frames,possible internal leaking?
0:00:03.225905403  2248 0x74e02a60 DEBUG           videodecoder
gstvideodecoder.c:3155:gst_video_decoder_clip_and_push_buf:<vdecoder>
Dropping frame due to QoS. start:0:00:01.828595512
deadline:0:00:01.828595512 earliest_time:0:00:01.897502021
0:00:03.244647270  2248   0xdf1690 DEBUG           videodecoder
gstvideodecoder.c:3408:gst_video_decoder_decode_frame:<vdecoder>
decoder frame list getting long: 12 frames,possible internal leaking?
0:00:03.249153987  2248 0x74e02a60 DEBUG           videodecoder
gstvideodecoder.c:2667:gst_video_decoder_prepare_finish_frame:<vdecoder>
sync timestamp 0:00:01.848735183 diff -0:00:01.848735183
0:00:03.249393674  2248 0x74e02a60 DEBUG           videodecoder
gstvideodecoder.c:3155:gst_video_decoder_clip_and_push_buf:<vdecoder>
Dropping frame due to QoS. start:0:00:01.848735183
deadline:0:00:01.848735183 earliest_time:0:00:01.897502021

I'm new to GStreamer, what does 'Dropping frame due to QoS' mean?
I assume this is the cause for the stuttering video playback - how can
it be fixed?

An additional question which is unclear to me: which is the correct
Gstreamer plugin to extract the H264 PES packes provided by the appsrc
to produce an elementary stream (ES) for the decoder?
Is it h264parse?

Thanks for any help,
Joerg

Am Fr., 27. Nov. 2020 um 19:30 Uhr schrieb Nicolas Dufresne
<nicolas at ndufresne.ca>:
>
> Le jeudi 26 novembre 2020 à 21:55 +0100, J. Krieg a écrit :
> > Hello,
> >
> > No ideas?
> > Could anyone help please?
> > Unfortunately I can’t figure out this by myself.
>
> By default, appsrc uses an open segment (start=0, end=infinity). That means your
> timestamp must match the running-time. The running time can be obtained like
> this:
>
>   clock = gst_pipeline_get_clock(pipeline)
>   if (clock) {
>         time_now = gst_clock_get_time(clock)
>         rt_time = time_now - gst_element_get_base_time (GST_ELEMENT
> (pipeline));
>    } else {
>         rt_time = GST_CLOCK_TIME_NONE; /* or 0 depending on your use case */
>    }
>
> If you have raw audio data, it might be easier to calculate the timestamp base
> ont he data lenght, starting from zero. Of if your data isn't live, you might
> also calculate timestamp using the framerate of a video (starting from 0 again).