Adding MetaData to GstBuffer

Mon Sep 27 11:00:16 UTC 2021

So I have created sample application with sender adding metadata and
receiver reading it. I am using templates for gst_Add_meta from here:
http://lifestyletransfer.com/how-to-add-metadata-to-gstreamer-buffer-in-python/

Relevant code from my sender and receiver as follows:
However, if I try to read meta data below in the same pipeline, it works.

rtpsink is udpsink and rtpsrc is udpsrc.

Sender:

#define AUDIO_CAPS "audio/x-raw,format=S16LE,channels=1"

gst_element_link_many(source, wavparse, audioconv,capsfilterAFX, rtpgstpay,
rtpsink, NULL)

ADD_PAD_PROBE(source, "src", Callback,
                    GST_PAD_PROBE_TYPE_BUFFER, nullptr);

GstPadProbeReturn Callback(GstPad *pad,
                                                         GstPadProbeInfo
*info,
                                                         gpointer
user_data) {
  GstBuffer *buffer = (GstBuffer *)info->data;

  GstMapInfo inmap = GST_MAP_INFO_INIT;

  g_print("\n data sending from source ");
  GstMapInfo map = GST_MAP_INFO_INIT;

  GstMetaMarking* meta = GST_META_MARKING_ADD(buffer);

  meta->in_timestamp = 99000;

  // Below works
    /*GstMetaMarking* meta1 = GST_META_MARKING_GET(buffer);
  if(meta1)
    g_print("\n data is %lu ", meta1->in_timestamp);
*/
  return GST_PAD_PROBE_OK;

}

Receiver:

// capsfilterAFX are same as sender AUDIO_CAPS

gboolean res=gst_element_link_many(rtpsrc, rtpgstdepay, capsfilterAFX,
audioconver,audioresample, fakesink,NULL);
g_assert(res==TRUE);

ADD_PAD_PROBE(capsfilterAFX, "src", Callback,
                    GST_PAD_PROBE_TYPE_BUFFER, nullptr);

GstPadProbeReturn Callback(GstPad *pad,
                           GstPadProbeInfo *info,
                           gpointer user_data) {

  GstBuffer *buffer = (GstBuffer *)info->data;
  //MaxineDSPipeline *pipeline = (MaxineDSPipeline *)user_data;
  GstMapInfo inmap = GST_MAP_INFO_INIT;

  GstMetaMarking* meta = GST_META_MARKING_GET(buffer);
  if(!meta)
    g_print("\n it is null ");
  else
    g_print("\n data received %lu", meta->in_timestamp);

  //g_print("\n data received");
  return GST_PAD_PROBE_OK;
}

However, my receiver, always get data null.

-
Thanks,
Roshan

On Mon, Sep 27, 2021 at 12:50 PM Eslam Ahmed <eslam.ahmed at avidbeam.com>
wrote:

> You are welcome!
>
> With gstrtppay/gstrtpdepay you should be able to serialize/deserialize the
> GstBuffer over the network no matter the type of data it contains.
> GstBuffer at this point could contain text, encoded/raw frame or an audio
> frame.
>
> Best Regards,
> Eslam Ahmed
>
>
> On Mon, Sep 27, 2021 at 9:09 AM Roshan Chaudhari <rgc183 at gmail.com> wrote:
>
>> Thank you Michael and Eslam!
>>
>> @Eslam, Seems like all the above options you mentioned are specific to
>> MPEG, or RTP?
>>
>> I am looking for a solution which could be applied to any data such as
>> text, audio, video data.
>>
>>
>> --
>> Thanks,
>> Roshan
>>
>> On Sun, Sep 26, 2021 at 2:21 PM Eslam Ahmed <eslam.ahmed at avidbeam.com>
>> wrote:
>>
>>> Roshan,
>>>
>>> Regarding Transport Stream, you can use the MPEG-TS container to contain
>>> your encoded stream as well as your metadata in a klv-format. Check this
>>> out:
>>> https://gstreamer-devel.narkive.com/GlIqaK1k/example-code-for-muxing-klv-meta-x-klv-with-mpegtsmux-plugins-bad-and-gstreamer-1-8-3
>>>
>>> Just make sure to push klv metadata every video frame otherwise the
>>> pipeline would stall. One useful thing to note is that your metadata need
>>> not be in a klv format, you can just push arbitrary bytes as long as you
>>> know how to interpret them after the demuxer at the receiver end.
>>>
>>> Another approach is to use payloaders (e.g. rtph264pay) and encode your
>>> metadata as RTP extension headers. see
>>> https://gstreamer.freedesktop.org/documentation/rtplib/gstrtpbuffer.html?gi-language=c#gst_rtp_buffer_add_extension_onebyte_header.
>>> If your metadata is per-frame basis, then most probably and based on the
>>> encoded frame size, one frame might end up having multiple RTP packets so
>>> you will need to handle where to insert your metadata. You can use the
>>> marker bit for that. see
>>> https://gstreamer.freedesktop.org/documentation/rtplib/gstrtpbuffer.html?gi-language=c#gst_rtp_buffer_get_marker
>>>
>>> Final approach which is much more complicated which will require you to
>>> create a new custom gstreamer media type (e.g. video/x-mytype) and insert
>>> your custom metadata via gst_buffer_add_meta() in the GstBuffer and use
>>> rtpgstpay/rtpgstdepay which guarantee to fully serialize/deserialize the
>>> GstBuffer over the network. You would also need to develop 2 gstreamer
>>> elements, one to prepend/append your metadata to your au-aligned H264 and
>>> the other to extract the metadata from video/x-mytype and restore the
>>> original stream.
>>>
>>> Hope that helps!
>>>
>>> Best Regards,
>>> Eslam Ahmed
>>>
>>>
>>> On Sat, Sep 25, 2021 at 10:00 PM Michael Gruner via gstreamer-devel <
>>> gstreamer-devel at lists.freedesktop.org> wrote:
>>>
>>>> Hello Roshan
>>>>
>>>> The GstMeta is not transmitted through network. If your want a standard
>>>> way to transmit metadata over the network along with audio and video you
>>>> may look into Transport Stream. This will allow you to insert your custom
>>>> metadata without breaking existing decoders. GStreamer already has support
>>>> for this.
>>>>
>>>> Another option may be to use a codec that allows you to insert custom
>>>> data within the compressed bitstream (like SEI in H264 or H265). That,
>>>> however is a bit more tricky to get right.
>>>>
>>>> Michael
>>>> www.ridgerun.con
>>>>
>>>> On 25 Sep 2021, at 08:24, Roshan Chaudhari via gstreamer-devel <
>>>> gstreamer-devel at lists.freedesktop.org> wrote:
>>>>
>>>> 
>>>> Hello,
>>>>
>>>> I am trying to append some metadata to the data transferred over udp
>>>> using gstreamer.
>>>>
>>>>
>>>>    1. I would like to know whether metadata in GstBuffer is
>>>>    transferred/retained when transferred over the network or it is only
>>>>    retained in that pipeline? I have tried using my custom metadata with
>>>>    gst_buffer_add_meta() at the udpsink, however, when I try to query it on
>>>>    udpsrc on other machine using gst_buffer_get_meta(), metadata is not
>>>>    present in GstBuffer.
>>>>    2.
>>>>    3. If this is not carried over the network, what would be the other
>>>>    way to add metadata? I could write custom plugin and append to actual data,
>>>>    so my custom encoder and decoder knows how to extract real data and pass it
>>>>    to next stage in the pipeline. But this way, it puts restriction on the
>>>>    receiver side to have decoder if my data contains metadata.
>>>>
>>>>
>>>> -
>>>> Thanks,
>>>> Roshan
>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210927/ceaee42f/attachment-0001.htm>