[Bug 743974] New: gl: rethink glfilter, glmixer, glupload, glcolorconvert, gldownload

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=743974
  GStreamer | gst-plugins-bad | git master

           Summary: gl: rethink glfilter, glmixer, glupload,
                    glcolorconvert, gldownload
    Classification: Platform
           Product: GStreamer
           Version: git master
        OS/Version: Linux
            Status: NEW
          Severity: enhancement
          Priority: Normal
         Component: gst-plugins-bad
        AssignedTo: gstreamer-bugs at lists.freedesktop.org
        ReportedBy: ystreet00 at gmail.com
         QAContact: gstreamer-bugs at lists.freedesktop.org
     GNOME version: ---


The problem I currently have with the current design of glfilter and glmixer is
that they are performing the functionality of multiple elements.  Along with
that, the formats supported are tied closely to the implementation of
glupload/glcolorconvert/gldownload.  The current pipeline inside glfilter is as
follows

glupload ! glcolorconvert ! gl operation/s ! glcolorconvert ! gldownload

with possible short cuts through each "element/object" to avoid unnecessary
processing that's not feature complete.  There's also the possibility of having
multiple upload stages as witnessed by the dmabuf support requiring the step
through EGLImage.  The other concern is that we do not expose EGLImage caps on
both src and sink pads and so elements like glcolorscale are not able to do
EGLImage passthrough.

I would like to split these different stages up into separate elements and put
them all into a bin.  In order to avoid losing the functionality of
GstBaseTransform/GstAggregator I propose the following split of elements

GstGLBaseFilter - current GstGLFilter without any format specifics.  It only
deals with the propagation of the GL context in the pipeline.  This would also
be the base class for glupload, glcolorconvert, gldownload, and any GL filter
element.
GstGLBase{Mixer,Aggregator,whatever} - current GstGLMixer without any format
specifics.  It only deals with the propagation of the GL context in the
pipeline like GstGLBaseFilter.
GstGLFilter - a bin like the above pipeline description that has a signal
"new-element" or somesuch that the 'subclass' can connect to to create an
element that performs their specific gl operation.  Or even multiple operations
in a row (like the sobel filters).
GstGL{Mixer,Aggregator,whatever} - similar idea to the new GstGLFilter.

Another advantage of the bin is the possibility of separating the
upload/download into separate threads and possibly separate gl contexts which
can improve throughput on certain platforms/configurations.

One concern with the proposed approach is current availability of using
glupload/glcolorconvert/gldownload from an independent/application perspective.
 I believe this use case to be a 'nice to have' rather than a hard requirement
in the "new age" due to the availability of glcolorscale and appsink/appsrc.

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.