Adding MetaData to GstBuffer

[Eslam Ahmed]

Roshan,

Regarding Transport Stream, you can use the MPEG-TS container to contain
your encoded stream as well as your metadata in a klv-format. Check this
out:
https://gstreamer-devel.narkive.com/GlIqaK1k/example-code-for-muxing-klv-meta-x-klv-with-mpegtsmux-plugins-bad-and-gstreamer-1-8-3

Just make sure to push klv metadata every video frame otherwise the
pipeline would stall. One useful thing to note is that your metadata need
not be in a klv format, you can just push arbitrary bytes as long as you
know how to interpret them after the demuxer at the receiver end.

Another approach is to use payloaders (e.g. rtph264pay) and encode your
metadata as RTP extension headers. see
https://gstreamer.freedesktop.org/documentation/rtplib/gstrtpbuffer.html?gi-language=c#gst_rtp_buffer_add_extension_onebyte_header.
If your metadata is per-frame basis, then most probably and based on the
encoded frame size, one frame might end up having multiple RTP packets so
you will need to handle where to insert your metadata. You can use the
marker bit for that. see
https://gstreamer.freedesktop.org/documentation/rtplib/gstrtpbuffer.html?gi-language=c#gst_rtp_buffer_get_marker

Final approach which is much more complicated which will require you to
create a new custom gstreamer media type (e.g. video/x-mytype) and insert
your custom metadata via gst_buffer_add_meta() in the GstBuffer and use
rtpgstpay/rtpgstdepay which guarantee to fully serialize/deserialize the
GstBuffer over the network. You would also need to develop 2 gstreamer
elements, one to prepend/append your metadata to your au-aligned H264 and
the other to extract the metadata from video/x-mytype and restore the
original stream.

Hope that helps!

Best Regards,
Eslam Ahmed


On Sat, Sep 25, 2021 at 10:00 PM Michael Gruner via gstreamer-devel <
gstreamer-devel at lists.freedesktop.org> wrote:

> Hello Roshan
>
> The GstMeta is not transmitted through network. If your want a standard
> way to transmit metadata over the network along with audio and video you
> may look into Transport Stream. This will allow you to insert your custom
> metadata without breaking existing decoders. GStreamer already has support
> for this.
>
> Another option may be to use a codec that allows you to insert custom data
> within the compressed bitstream (like SEI in H264 or H265). That, however
> is a bit more tricky to get right.
>
> Michael
> www.ridgerun.con
>
> On 25 Sep 2021, at 08:24, Roshan Chaudhari via gstreamer-devel <
> gstreamer-devel at lists.freedesktop.org> wrote:
>
> 
> Hello,
>
> I am trying to append some metadata to the data transferred over udp using
> gstreamer.
>
>
>    1. I would like to know whether metadata in GstBuffer is
>    transferred/retained when transferred over the network or it is only
>    retained in that pipeline? I have tried using my custom metadata with
>    gst_buffer_add_meta() at the udpsink, however, when I try to query it on
>    udpsrc on other machine using gst_buffer_get_meta(), metadata is not
>    present in GstBuffer.
>    2.
>    3. If this is not carried over the network, what would be the other
>    way to add metadata? I could write custom plugin and append to actual data,
>    so my custom encoder and decoder knows how to extract real data and pass it
>    to next stage in the pipeline. But this way, it puts restriction on the
>    receiver side to have decoder if my data contains metadata.
>
>
> -
> Thanks,
> Roshan
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210926/c635b5b3/attachment-0001.htm>