Adding MetaData to GstBuffer

[Roshan Chaudhari]

Thank you Michael and Eslam!

@Eslam, Seems like all the above options you mentioned are specific to
MPEG, or RTP?

I am looking for a solution which could be applied to any data such as
text, audio, video data.


--
Thanks,
Roshan

On Sun, Sep 26, 2021 at 2:21 PM Eslam Ahmed <eslam.ahmed at avidbeam.com>
wrote:

> Roshan,
>
> Regarding Transport Stream, you can use the MPEG-TS container to contain
> your encoded stream as well as your metadata in a klv-format. Check this
> out:
> https://gstreamer-devel.narkive.com/GlIqaK1k/example-code-for-muxing-klv-meta-x-klv-with-mpegtsmux-plugins-bad-and-gstreamer-1-8-3
>
> Just make sure to push klv metadata every video frame otherwise the
> pipeline would stall. One useful thing to note is that your metadata need
> not be in a klv format, you can just push arbitrary bytes as long as you
> know how to interpret them after the demuxer at the receiver end.
>
> Another approach is to use payloaders (e.g. rtph264pay) and encode your
> metadata as RTP extension headers. see
> https://gstreamer.freedesktop.org/documentation/rtplib/gstrtpbuffer.html?gi-language=c#gst_rtp_buffer_add_extension_onebyte_header.
> If your metadata is per-frame basis, then most probably and based on the
> encoded frame size, one frame might end up having multiple RTP packets so
> you will need to handle where to insert your metadata. You can use the
> marker bit for that. see
> https://gstreamer.freedesktop.org/documentation/rtplib/gstrtpbuffer.html?gi-language=c#gst_rtp_buffer_get_marker
>
> Final approach which is much more complicated which will require you to
> create a new custom gstreamer media type (e.g. video/x-mytype) and insert
> your custom metadata via gst_buffer_add_meta() in the GstBuffer and use
> rtpgstpay/rtpgstdepay which guarantee to fully serialize/deserialize the
> GstBuffer over the network. You would also need to develop 2 gstreamer
> elements, one to prepend/append your metadata to your au-aligned H264 and
> the other to extract the metadata from video/x-mytype and restore the
> original stream.
>
> Hope that helps!
>
> Best Regards,
> Eslam Ahmed
>
>
> On Sat, Sep 25, 2021 at 10:00 PM Michael Gruner via gstreamer-devel <
> gstreamer-devel at lists.freedesktop.org> wrote:
>
>> Hello Roshan
>>
>> The GstMeta is not transmitted through network. If your want a standard
>> way to transmit metadata over the network along with audio and video you
>> may look into Transport Stream. This will allow you to insert your custom
>> metadata without breaking existing decoders. GStreamer already has support
>> for this.
>>
>> Another option may be to use a codec that allows you to insert custom
>> data within the compressed bitstream (like SEI in H264 or H265). That,
>> however is a bit more tricky to get right.
>>
>> Michael
>> www.ridgerun.con
>>
>> On 25 Sep 2021, at 08:24, Roshan Chaudhari via gstreamer-devel <
>> gstreamer-devel at lists.freedesktop.org> wrote:
>>
>> 
>> Hello,
>>
>> I am trying to append some metadata to the data transferred over udp
>> using gstreamer.
>>
>>
>>    1. I would like to know whether metadata in GstBuffer is
>>    transferred/retained when transferred over the network or it is only
>>    retained in that pipeline? I have tried using my custom metadata with
>>    gst_buffer_add_meta() at the udpsink, however, when I try to query it on
>>    udpsrc on other machine using gst_buffer_get_meta(), metadata is not
>>    present in GstBuffer.
>>    2.
>>    3. If this is not carried over the network, what would be the other
>>    way to add metadata? I could write custom plugin and append to actual data,
>>    so my custom encoder and decoder knows how to extract real data and pass it
>>    to next stage in the pipeline. But this way, it puts restriction on the
>>    receiver side to have decoder if my data contains metadata.
>>
>>
>> -
>> Thanks,
>> Roshan
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210927/29e5a75c/attachment-0001.htm>