Creating a dedicated per frame metadata stream?

[pisymbol .]

Here is what I'd like to do:

I have a *live* source that is capturing video. What I'd like to do is
create a multistream file, say an MLV, where one stream is the actually
video while another is a data stream essentially of per fram metadata
(think timestamps and other sensory data at the time of capture).

I know vendors like GoPro do this: They basically create I think a
complimentary audio stream that is actually metadata but I'm not exactly
sure how. I've also seen other smaller third-paty do something similar with
video overlay.

Has anyone done this before using GStreamer? If so, can you give me a
little guidance on how that would work? Is this a custom plugin or are
there exiting primitives I could exploit?

Thanks!

-aps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190823/13a7be52/attachment.html>