Identifying h264 frame types?

[will at mcelderry.com]

On 2024-01-25 13:54, Tim-Philipp Müller via gstreamer-devel wrote:
> Hi,
> 
>> Short version:
>> 
>> Is there a way to identify if a sample/frame provided to an
>> appsink
>> 
>> has been produced from an h264 I-frame (or P-frame/B-frame)?
> 
> Encoded H.264 frames coming out of qtdemux should have the DELTA_UNIT
> flag set on buffers if it's a P/B frame, and should have the
> DELTA_UNIT flag cleared (not set) if it's a key/IDR frame. This will
> be based on information in the container.
> 
> After a decoder you can't know if a decoded raw video frame was
> generated from a keyframe or not. (If that flag happens to be set
> right in some cases that would likely be an accident, not by design,
> and you probably can't rely on that).
> 
> Cheers
>  Tim


Hi Tim,

Thanks for your feedback!

I will explore if it is possible to listen to the frame's metadata after 
the qtdemux (using one of: a probe / an identity element with handoff / 
tee ! appsink ) to see if I can use the information from the qtdemux to 
build my lookup table with one pass.

Thanks Stephane too! I'll explore ESExtractor if I'm struggling.

All the best,

Will.