RTMP Metadata Stream For Augmented Reality

[Brendan Lockhart]

Hey all,

I'm developing an augmented reality app which uses OpenCV within a
Gstreamer pipeline to detect certain objects and stream those objects
positions within the video stream to clients of an nginx-rtmp server.

I've finished building the detection element, but I'm having a hard time
figuring out how to actually stream the position information. Ideally, I'd
send that data as a text stream, but it appears that rtmpsink/rtmpsrc don't
support subtitles or onTextData events. As a result, I'm now evaluating
different possible solutions for how to transmit this data.

The ideas that I'm currently evaluating are:

   1. Serialize the position data into a video or audio stream, and have
   the clients deserialize that data into positions. I think that this could
   work, but the compression done might cause some unintended effects
   2. Send the data out-of-band, possibly using a tcp sink and
   gst-rtsp-server. This one seems the most likely to work, but I worry that
   synchronization would then be very difficult, and I'm not really sure where
   to begin
   3. Adapt Gstreamer's rtmp elements to support onTextData. Being the nerd
   that I am, I would love to do this, but unfortunately I'm a noob at C so
   this would probably take me a huge amount of time and be too low quality to
   be of much value to the community.

Are there any options for this that I've missed? Any suggestions or advice
would be more than appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20170612/3bd73d39/attachment.html>