Adding typefind function for a new Gst Type, "tensors"

[MyungJoo Ham]

Dear Gstreamer Developers,


We have been working on developing Gstreamer plugins to handle
general neural network models implemented with on-the-shelf frameworks
(such as Tensorflow or Caffe) as media filters.


I have a few questions on how to register a new custom "typefind" for
a new non-media type, "tensor/tensors".



Because the "sources" are general media streams, such as "camera", "mic",
or "media files", the to-be-proposed type, "other/tensor" and
"other/tensors", do not have their file format, yet.
Their meta data are handled as pad-capabilities and
the buffer between elements contains the tensor data only
(similar to what video/x-raw does)

However, assuming that we need to register "typefind",
later to be considered to become "good plugin" some day,
it appears that we need to define headers as if they are
stored as files. (for debugging and testing purpose, we may need this anyway)



Q1. Is it ok to define "other/tensor(s)" type to have some headers
(including dimensions of tensors) when it is stored in file and
not to have any meta data in inter-gst-element transfer (like video/x-raw)?

Q2. If Q1 == NO, is it ok to define "other/tensorsave" to represent
the saved file format for tensors and use "other/tensor(s)" for
video/x-raw-like handling?

Q3. Or is it ok not to register any and possible to upstream to
gst-plugins-good with these new media types (other/tensor and other/tensors)?



As I've mentioned in Gstreamer-IRC back in March (sorry, I was away from
this project for a while and returned), we've finally got internal approval
to fully open sources (In LGPL-2.1.)
We will be migrating the code to github.com/nnsuite soon, probably in this week.


The full migration requires CI/CD systems to be migrated as well,
so the migration of developing process might take some time.



We hope, someday, we can upstream the plugins to gst-plugins-good. :)





Cheers,
MyungJoo

--

MyungJoo Ham (함명주), Ph.D.
Autonomous Machine Lab., AI Center, Samsung Research.
Cell: +82-10-6714-2858