Pointer to I-frame extra data in H.264?

[Nicolas Dufresne]

Le jeudi 25 février 2021 à 00:59 -0600, Javiku a écrit :
> Hi,
> 
> I am working on a project where I need to feed some application functions
> with the output of an H.264 encoder (v4l2h264enc). Those functions expect a
> pointer to the encoded data and a second pointer to the "extra data" from
> the I-Frames. This extra data is defined in the application documentation as
> follows:

v4l2h264enc produces byte-stream (Annex B) stream, which means the headers are
inline. To get "extra-data" ala FFMPEG, you need to convert that to AVCc format.
The data will then be in the caps, under the field name codec_data with the type
GstBuffer. Something like:

  ... ! v4l2h264enc ! h264parse ! video/x-h264,profile=main,level=4.1,stream-format=avc ! ...

> 
> "The extra ancillary data required to configure an H.264 codec must
> correctly be provided. This must exist for all I-
> frames. The structure of this data should contain concatenated NAL units in
> Annex B format, along with their start
> codes. For H.264, they are the SPS and PPS NAL units."
> 
> So far I managed to create an appsink, link it to a tee object and use a
> callback to extract a buffer every time a new sample is generated. Then I
> mapped that buffer to get the pointer to the data (map.data) and de data
> size (map.size) as well. But I could not get any further because the
> GstMapInfo structure does not provide any specific field for I-frames extra
> data and I could not find it in the GstSample or GstBuffer structures
> either. By the way, I am checking if the GST_BUFFER_FLAG_DELTA_UNIT flag is
> not set to find the I-frames, which seems to work fine because the buffers
> are way bigger when that happens (once every 30 frames at 30 fps).
> 
> I was wondering if there is a specific structure member where I could find
> this extra data or if I should look for it somewhere else... maybe packed
> within the data block? I had a look at the first bytes of my I-frames and
> they start with 0x00 0x00 0x00 0x01 0x07 0x42 0x40 0x28... as other non
> I-frames do (maybe P-frames). Could that data be the extra data (SPS and PPS
> NAL units) anyways? The GST_BUFFER_FLAG_HEADER flag is never set though.
> 
> Thanks a lot for your help, best regards,
> Javiku
> 
> 
> 
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel