Accurately seeking to frames?

[Nicolas Dufresne]

Le mardi 16 janvier 2024 à 16:09 +0000, Will McElderry via gstreamer-devel a
écrit :
> Hi Nicolas,
> 
> 
> Thanks for your fast & authoritative reply!
> 
> 
> I've still got an issue with accurately seeking to a frame where seeking 
> to the frame's (stream-time equivalent of) PTS does not give the target 
> frame (or even the following frame - sometimes I get the preceding frame)
> 
> I believe the issue is caused by the data: a combination of variable 
> frame rate with fixed frame duration (the duration is generally shorter 
> than the inter-frame PTS intervals)

For variable rate streams, there is nothing in the ISOMP4 demuxer to help you.
Best to just pause the stream, and step (using the step operation) to the frame
number you want. It does mean decoding frames from the start.

To optimize this further, you could create a map of PTS/frame_number pairs as an
offline reprocessing. With exact TS, you should be able to seek accurately.

Nicolas

> 
> 
> 1. Could constant duration combined with variable inter-frame PTS 
> explain 'weird seeking'? (I suspect it does, but could be guessing the 
> wrong solution)  (if useful I can illustrate this with actual 
> PTS/duration,seeks etc from a file)

> 
> 2. If so, is there a minimal impact method to fix up a file with 
> incorrect durations to match the frame PTS times?  (ideally not 
> transcoding, just updating meta-data values)
> 
>      [container was mp4mux, video stream is h264 captured from an RTSP 
> camera]
> 
>      (? Is that even possible, or is frame duration always just one 
> number for the entire stream?)
> 
> 3. What does frame duration mean?  (& what should it be for a file with 
> variable frame rate?)
> 
>      3.1. What is a frame duration conceptually and how does it relate 
> to the PTS?
> 
>              - the duration to show a frame, following the PTS
> 
>              - the duration to show a frame, before the PTS
> 
>              - the duration to show a frame, centered about the PTS
> 
>      3.2. should the first frame (frame 0) in the file have a duration 
> equal to:
> 
>              - frame 0's PTS
> 
>              - the interval between frame 0 and frame 1 PTS
> 
>              - frame 1's PTS
> 
>      3.3. Subsequent frame (frame n) durations should be:
> 
>              - the interval between frame n-1 and frame n PTS
> 
>              - the interval between frame n and and n+1 PTS
> 
>      3.4. In a manner similar to PTS needing to be translated by the 
> Segment to get the PTS time in the stream, does duration need any 
> translation by the Segment?
> 
> Thanks again in advance!
> 
> 
> Will.
> 
> 
> R.E. Ken's comment: Thanks for your contribution - unfortunately that's 
> not easy to answer as it all depends on your pipeline and seems like a 
> totally different discussion.  If you want to discuss further, please 
> start another thread and I'll tell you the limited amount I think I 
> can.  All the best!
>