[gst-devel] Decoded video frame timings

[Darren Staples]

Hi,

I'm decoding video frames (various file formats and codecs) and converting
them into YUV420 format so that I can perform image processing on them.
And I have the commercial Fluendo codec pack installed.

The pipeline looks like this: uridecodebin -> queue -> ffmpegcolorspace ->
tee -> queue -> appsink
YUV420 conversion is done in the ffmpegcolorspace component, and the other
source pad of the tee is for a future feature.
This is built up dynamically in C, not on a command line.

My problem is that I need to extract the frame timings for each of the video
frames being processed, and I'm seeing a large variance in this.
For example, one of my video files is a H.264 MP4, 82s long, 2057 frames and
plays at 25 fps, so the average frame interval is 40ms.
When I recover a frame from the appsink component I use the
gst_element_query_position() function to get the timing for that frame.
And what I'm seeing is that the timing is highly variable, even from one run
to the next.

e.g. the time between frames for the first 32 frames for run 1:
0: 1020226us
1: 0us
2: 0us
3: 0us
4: 0us
5: 0us
6: 0us
7: 0us
8: 0us
9: 0us
10: 0us
11: 0us
12: 0us
13: 0us
14: 0us
15: 0us
16: 0us
17: 0us
18: 0us
19: 0us
20: 0us
21: 0us
22: 0us
23: 0us
24: 0us
25: 0us
26: 19774us
27: 40000us
28: 40000us
29: 1022947us
30: 46440us
31: 23220us

the first 32 frames for run 2:
0: 53151us
1: 0us
2: 26849us
3: 40000us
4: 40000us
5: 40000us
6: 40000us
7: 40000us
8: 40000us
9: 40000us
10: 40000us
11: 40000us
12: 40000us
13: 40000us
14: 40000us
15: 40000us
16: 40000us
17: 40000us
18: 40000us
19: 40000us
20: 1011428us
21: 46440us
22: 46440us
23: 23220us
24: 46440us
25: 46440us
26: 46440us
27: 23219us
28: 46440us
29: 46440us
30: 46440us
31: 23220us

Why do so many of the frames in the first run above have zero time delta
between them ?
Is this a consequence of the file being in H.264/mp4 format ? And if, so is
there an easy way to 'smooth' out the frame timings ?
Or would you suggest I do this 'manually' by approximating/averaging the
timing based on the duration and no. of frames ?

Thanks !
Darren
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20110112/ca76bc1f/attachment.htm>