Adding MetaData to GstBuffer

[Eslam Ahmed]

You are welcome!

With gstrtppay/gstrtpdepay you should be able to serialize/deserialize the
GstBuffer over the network no matter the type of data it contains.
GstBuffer at this point could contain text, encoded/raw frame or an audio
frame.

Best Regards,
Eslam Ahmed


On Mon, Sep 27, 2021 at 9:09 AM Roshan Chaudhari <rgc183 at gmail.com> wrote:

> Thank you Michael and Eslam!
>
> @Eslam, Seems like all the above options you mentioned are specific to
> MPEG, or RTP?
>
> I am looking for a solution which could be applied to any data such as
> text, audio, video data.
>
>
> --
> Thanks,
> Roshan
>
> On Sun, Sep 26, 2021 at 2:21 PM Eslam Ahmed <eslam.ahmed at avidbeam.com>
> wrote:
>
>> Roshan,
>>
>> Regarding Transport Stream, you can use the MPEG-TS container to contain
>> your encoded stream as well as your metadata in a klv-format. Check this
>> out:
>> https://gstreamer-devel.narkive.com/GlIqaK1k/example-code-for-muxing-klv-meta-x-klv-with-mpegtsmux-plugins-bad-and-gstreamer-1-8-3
>>
>> Just make sure to push klv metadata every video frame otherwise the
>> pipeline would stall. One useful thing to note is that your metadata need
>> not be in a klv format, you can just push arbitrary bytes as long as you
>> know how to interpret them after the demuxer at the receiver end.
>>
>> Another approach is to use payloaders (e.g. rtph264pay) and encode your
>> metadata as RTP extension headers. see
>> https://gstreamer.freedesktop.org/documentation/rtplib/gstrtpbuffer.html?gi-language=c#gst_rtp_buffer_add_extension_onebyte_header.
>> If your metadata is per-frame basis, then most probably and based on the
>> encoded frame size, one frame might end up having multiple RTP packets so
>> you will need to handle where to insert your metadata. You can use the
>> marker bit for that. see
>> https://gstreamer.freedesktop.org/documentation/rtplib/gstrtpbuffer.html?gi-language=c#gst_rtp_buffer_get_marker
>>
>> Final approach which is much more complicated which will require you to
>> create a new custom gstreamer media type (e.g. video/x-mytype) and insert
>> your custom metadata via gst_buffer_add_meta() in the GstBuffer and use
>> rtpgstpay/rtpgstdepay which guarantee to fully serialize/deserialize the
>> GstBuffer over the network. You would also need to develop 2 gstreamer
>> elements, one to prepend/append your metadata to your au-aligned H264 and
>> the other to extract the metadata from video/x-mytype and restore the
>> original stream.
>>
>> Hope that helps!
>>
>> Best Regards,
>> Eslam Ahmed
>>
>>
>> On Sat, Sep 25, 2021 at 10:00 PM Michael Gruner via gstreamer-devel <
>> gstreamer-devel at lists.freedesktop.org> wrote:
>>
>>> Hello Roshan
>>>
>>> The GstMeta is not transmitted through network. If your want a standard
>>> way to transmit metadata over the network along with audio and video you
>>> may look into Transport Stream. This will allow you to insert your custom
>>> metadata without breaking existing decoders. GStreamer already has support
>>> for this.
>>>
>>> Another option may be to use a codec that allows you to insert custom
>>> data within the compressed bitstream (like SEI in H264 or H265). That,
>>> however is a bit more tricky to get right.
>>>
>>> Michael
>>> www.ridgerun.con
>>>
>>> On 25 Sep 2021, at 08:24, Roshan Chaudhari via gstreamer-devel <
>>> gstreamer-devel at lists.freedesktop.org> wrote:
>>>
>>> 
>>> Hello,
>>>
>>> I am trying to append some metadata to the data transferred over udp
>>> using gstreamer.
>>>
>>>
>>>    1. I would like to know whether metadata in GstBuffer is
>>>    transferred/retained when transferred over the network or it is only
>>>    retained in that pipeline? I have tried using my custom metadata with
>>>    gst_buffer_add_meta() at the udpsink, however, when I try to query it on
>>>    udpsrc on other machine using gst_buffer_get_meta(), metadata is not
>>>    present in GstBuffer.
>>>    2.
>>>    3. If this is not carried over the network, what would be the other
>>>    way to add metadata? I could write custom plugin and append to actual data,
>>>    so my custom encoder and decoder knows how to extract real data and pass it
>>>    to next stage in the pipeline. But this way, it puts restriction on the
>>>    receiver side to have decoder if my data contains metadata.
>>>
>>>
>>> -
>>> Thanks,
>>> Roshan
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210927/23760e8f/attachment.htm>