AW: gst_adapter_take_buffer and gst_buffer_copy_into problem

[Thornton, Keith]

Hi,
00 00 01 67 4D 40 29 96 54 03 C0    >..|....gM@).T.@<mailto:..|....gM@).T.@>
00000040: 11 3F 2A 00 00 00 01 68 EE 38 80
Is an SPS NAL.
Gruesse

Von: gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org> Im Auftrag von Andreas Corneliussen
Gesendet: Donnerstag, 21. November 2019 10:34
An: gstreamer-devel at lists.freedesktop.org
Betreff: gst_adapter_take_buffer and gst_buffer_copy_into problem

Hi

I have an embedded target where I am unable to upgrade the gstreamer installation higher than 12.3 due to a kernel with proprietary drivers not available in recent versions and a resulting problem with updating yocto to a recent version as well.

On this target I need to rtp a h264 stream from a camera interfaced using the v4l2src. I therefore use the following pipeline to determine if the data sent on RTP is correct:

gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-h264, width=1920, height=1080, framerate=10/1 ! tee name=t t. ! queue ! filesink location=/home/dtc/testfilev4l2.dat t. ! queue ! rtph264pay name=pay0 ! filesink location=/home/dtc/testfilepay.dat

When I then look at the binary file testfilepay.dat I see the following:

 Offset: 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000: 80 60 28 B5 40 5B 2F CD 7A 3E 18 90 67 4D 40 29    .`(5@[/Mz>..gM@<mailto:5@[/Mz%3e..gM@>)
00000010: 96 54 03 C0 11 3F 2A 80 60 28 B6 40 5B 2F CD 7A    .T. at .?*.`(6@[/Mz<mailto:.T. at .?*.%60(6@[/Mz>
00000020: 3E 18 90 68 EE 38 80 80 60 28 B7 40 5B 2F CD 7A    >..hn8..`(7@[/Mz
00000030: 3E 18 90 7C 85 00 00 01 67 4D 40 29 96 54 03 C0    >..|....gM@).T.@<mailto:..|....gM@).T.@>
00000040: 11 3F 2A 00 00 00 01 68 EE 38 80 00 00 00 01 65    .?*....hn8.....e
00000050: 88 80 40 02 FF FE 04 18 70 29 1B 5D 15 CC 05 E9    .. at ..~..p).].L.i<mailto:.. at ..~..p).].L.i>

What I see from the about it that the two first NALs of the h264 stream is parsed correctly and the 00 00 00 01 h264 header is removed and replaced with an RTP header (marked in read above). However, for the third NAL (an 0x65 NAL) it seems that it starts copying from the input stream 27 bytes too early resulting in both the two first NALs to be duplicated into the stream where the second time they appear they are prepended with the h264 header 00 00 00 01 or 00 00 01 and the header of the third NAL to be copied incorrectly (all this is marked with pink above). In other words all the 27 bytes marked with pink above should be removed.

I have then investigated in the rtph264payload source code how this happens and what I have found is that the payload is doing the following to each input buffer:

1. The input buffer to the element in pushed into an GstAdapter
2. Then the full GstAdapter is searched through in order to determine the start position of all the NALs
3. Then for each of the NALs a buffer is taken from from the adapter with gst_adapter_take_buffer
4. These buffers are then parsed to determine which type of NAL is contained within and how big the buffer is
5. If the buffer size is below the mtu size of the system:
    5.1. Make a new GstBuffer with the correct header
    5.2. Append the buffer taken from the adapter to the new GstBuffer with gst_buffer_append()
    5.3. Send that new buffer out on the src pad
6. If the buffer size is above the mtu size of the system:
    6.1. Make a new GstBuffer with the correct header
    6.2. Copy the correct part of the buffer taken from the adapter into the new buffer using gst_buffer_copy_into()
    6.3. Send the new buffer out on the src pad
    6.4. Determine if all the buffer taken from the GstAdapter has been copied to output buffers and if not then go back to 6.1

I have also determined that the problem seems to be that the gst_buffer_copy_into() seems to disregard that the buffer taken from the GstAdapter does not start at the start of the adapter. In other words, when we use gst_buffer_copy_info() to copy from the buffer taken from the GstAdapter with an offset of zero it starts at offset 0 relative to the start of the data pushed into the adapter even though 27 bytes have been taken or flushed from the adapter before this.

Does anyone know of a way to make the function gst_buffer_copy_into respect the offset of the input buffer instead of starting at the start of the adapter?

Kind regards

Andreas Corneliussen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20191121/8e7e0759/attachment-0001.html>