AW: Pointer to I-frame extra data in H.264?

[Thornton, Keith]

Hi,
The first four bytes in your example are the start code. The next byte indicates the NAL unit type.
Take a look at the description provided in e.g. https://yumichan.net/video-processing/video-compression/introduction-to-h264-nal-unit/ As you can see in his example, there are more than one NAl unit in the buffer. This is encoder specific.
Gruesse

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org> Im Auftrag von Javiku
Gesendet: Donnerstag, 25. Februar 2021 07:59
An: gstreamer-devel at lists.freedesktop.org
Betreff: Pointer to I-frame extra data in H.264?

Hi,

I am working on a project where I need to feed some application functions with the output of an H.264 encoder (v4l2h264enc). Those functions expect a pointer to the encoded data and a second pointer to the "extra data" from the I-Frames. This extra data is defined in the application documentation as
follows:

"The extra ancillary data required to configure an H.264 codec must correctly be provided. This must exist for all I- frames. The structure of this data should contain concatenated NAL units in Annex B format, along with their start codes. For H.264, they are the SPS and PPS NAL units."

So far I managed to create an appsink, link it to a tee object and use a callback to extract a buffer every time a new sample is generated. Then I mapped that buffer to get the pointer to the data (map.data) and de data size (map.size) as well. But I could not get any further because the GstMapInfo structure does not provide any specific field for I-frames extra data and I could not find it in the GstSample or GstBuffer structures either. By the way, I am checking if the GST_BUFFER_FLAG_DELTA_UNIT flag is not set to find the I-frames, which seems to work fine because the buffers are way bigger when that happens (once every 30 frames at 30 fps).

I was wondering if there is a specific structure member where I could find this extra data or if I should look for it somewhere else... maybe packed within the data block? I had a look at the first bytes of my I-frames and they start with 0x00 0x00 0x00 0x01 0x07 0x42 0x40 0x28... as other non I-frames do (maybe P-frames). Could that data be the extra data (SPS and PPS NAL units) anyways? The GST_BUFFER_FLAG_HEADER flag is never set though.

Thanks a lot for your help, best regards, Javiku



--
Sent from: https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgstreamer-devel.966125.n4.nabble.com%2F&data=04%7C01%7C%7C92f95a8be4ca4ea362ec08d8d97c9112%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637498476320172732%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=wPU2%2FmvF8fupieZPYh4G75VUJ%2BWGGUZ79hf1eaWIMkY%3D&reserved=0
_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&data=04%7C01%7C%7C92f95a8be4ca4ea362ec08d8d97c9112%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637498476320172732%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CRkIZ5pVCa1qW7caTWdGkOxS%2F2BLS2aEl%2B3FE%2FQFqvk%3D&reserved=0