glimagesink and how it likes to do things

Fri Aug 8 06:32:47 PDT 2014

On 08/08/14 21:41, Lasse Laursen wrote:
> Thanks a ton for the answer Matthew - although your answers opens up a
> few more questions, if you don't mind?
>
> On 08-08-2014 20:12, Matthew Waters wrote:
>> On 08/08/14 20:27, Lasse Laursen wrote:
>>> Looking closer at available plug-ins, there's also something called
>>> GstGLUpload - which says that it uploads to GL Textures... Is this
>>> supposed to be used as a sink? Can it?
>> GstGLImageSink is the only sink currently provided by the opengl
>> plugins.  In 1.4, GstGLUpload is now a helper object (not an element)
>> available within libgstgl for elements/applications to use.  All opengl
>> elements implicitly upload/download as required by the pipeline using it
>> and its counterpart, GstGLDownload.
>
> Ah - so if I understand you correctly - it's completely possible (and
> even ok) to use an opengl element (such as the GstGLShader element) in
> a pipeline, and then have the final result arrive at an appsink, and
> not the gstglimagesink?

Correct.

> You say all elements implicitly upload/download as required, so am I
> to understand that I could define a pipeline as follows "uridecodebin
> uri=<some file> ! GlShader ! appsink name=sink caps=<some caps>", and
> pass it to 'gst_parse_launch', and it'd figure out to insert the
> upload/download elements around the shader as required?

Correct although the upload/download are not in separate elements
anymore, it is provided by the GstGLUpload object in the libgstgl
library.  The docs for the libgstgl library that the opengl elements use
is
http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad-libs/html/gl.html
which are up to date.

>>> I've also just learned that if I wanted to use the GstGLShader object,
>>> I am forced to make use of the GstGLImageSink? Would GstGLUpload also
>>> be acceptable here?
>> GstGLShader (the element), like all opengl elements will upload/download
>> data as required by the surrounding elements.  So you could have any
>> other non-GL element after it and it should still work.
>>
>>> Regards,
>>> Lasse
>>>
>>> On 08-08-2014 19:09, Lasse Laursen wrote:
>>>> Hey y'all,
>>>>
>>>> I know this is kind of a beaten horse, but given the recent release
>>>> of the 1.4 version of gStreamer I thought I'd pull out a stick and
>>>> give it a few more whacks.
>>>>
>>>> Today I finally got around to working out how to properly use an
>>>> appsink to copy gstreamers video output directly into an OpenGL
>>>> texture. This works pretty well for now - but I'm constantly trying
>>>> to work out, if there's a way of bypassing having to copy the data
>>>> manually and just have gstreamer throw it into an OpenGL texture
>>>> for me.
>> GstGLUpload is designed for this use case :)
>
> In your first reply, you mention that GlImageSink is currently the
> *only* sink provided by the opengl plugins, does this mean that I'm
> not allowed to have a pipeline end in a GstGLUpload, if all I want is
> the data to end in a texture I am managing? Looking at the
> documentation, for GstGLUpload, I'm in doubt if this is even possible?
> That is to say - that I specify the texture ID?

The 0.10 GstGLUpload was a filter element that transformed from raw
video frames into texture handles so it is/was not possible to use
gstglupload as a sink.  As for specifying the texture id, that is
currently not possible with the current API.

> You ask at the end why I want to set the texture ID - yes -
> essentially I'm using the Cinder framework, and I'd like to integrate
> GStreamers output in a simple GUI I'm building. To have it play well
> with Cinder, I'd need to use cinders own Texture object, which - as
> far as I can tell either requires me to give a fixed texture ID to
> gstreamer where I'd like the data to end up, or retrieve a texture ID
> from GStreamer and build the cinder texture object using it.

One option is to use
http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad-libs/html/gst-plugins-bad-libs-gstglmemory.html#gst-gl-memory-copy-into-texture
however that involves a gpu side copy that will not work for all
textures on all platforms (RGBA is fine though).

> ----
>
> Just to reiterate - My current implementation scenario is, I use
> gstreamer to build a pipeline which currently includes 0 GL elements.
> It ends with an Appsink, from which I directly copy data into a
> Texture for every received sample. Seems to work ok for now.
>
> 2 Potential scenarios:
>
> 1 - A pipeline with 0 GL elements - except potentially the last one.
> Is GstGLUpload the way to go to have the data end up in a texture for
> me to use? Any example code?

I would build a pipeline looking like:
... ! glcolorscale ! appsink

The upload/download would be done by the glcolorscale element and the
specific caps (with features) on either side of it.  The appsink would
upload the buffer using map and wrap the GstGLMemory inside the
GstBuffer's you receive into Cinder's Texture object.

If you run 'gst-inspect-1.0 glcolorscale', you'll see that one of the
caps supported on the src and sinkpads include the 'memory:GLMemory'
caps feature.  That means that the GstBuffers that glcolorscale
consumes/produces will contain GstGLMemory's which contain GL texture
handles.

> 2 - A pipeline with 1 or more GL elements, most likely the GstGLShader
> object. Is it possible to access the handled data that's stored in a
> texture on the GPU? Or is it much easier to let it get handed back to
> the CPU, just to upload it again?

... ! glshader ! appsink
Exactly the same idea as in 1.  You would have an appsink that would
transform the GstGLMemory into Cinder's Texture.

>>
>>>> The glimagesink looked promising, despite it's current 'bad' status
>>>> and... 0.10 reference manual? I take it, this is the most up-to-date
>>>> doc currently:
>>>>
>>>> http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-gl-plugins/html/
>>>>
>>>>
>>>> (although it probably should eventually be merged into the 'bad'
>>>> folder given it's status?)
>> There are docs in the -bad repository however they don't seem to have
>> been updated on/uploaded to the server.
>>
>>>> Anyway - I had a closer look at the following example:
>>>>
>>>> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/generic/cube/main.cpp
>>>>
>>>>
>>>>
>>>> It seems as though the client callback receives a texture
>>>> reference... Is there anyway of specifying a texture to which one
>>>> would like the glimagesink to write, instead of receiving a reference
>>>> to one in every callback? If not - is it possible to somehow
>>>> intercept the texture ID earlier - and are there any guarantees that
>>>> it doesn't change at some point down the line?
>> The exact behaviour will depend on the specific elements involved.  The
>> two possibilities are
>> 1. glimagesink uploads the incoming video frame -> texture id will
>> change every draw call (performance gain is possible here although only
>> to the second case)
>> 2. glimagesink displays upstream's GL texture and is using a bufferpool
>> -> texture id will loop between a certain amount of texture id's
>> depending on the lifetime of the buffers.  This is so that it is
>> possible to upload the previous texture while the next texture is being
>> displayed/used.  The other catch is that the basesink class by default
>> holds on to the last buffer rendered until the next buffer arrives.
>>
>> Any existence of the texture id outside of the callback is not really
>> guaranteed.  Is there any particular reason you would like to set the
>> texture id for upload?
>>
>>>> Regards,
>>>> Lasse
>>>>
>>>> -- 
>>>> Lasse Farnung Laursen
>>>> Post.doc at the University of Tokyo
>>>> www.lasselaursen.com <http://www.lasselaursen.com>
>>>> FB: lasse.laursen <https://www.facebook.com/lasse.laursen>
>>>> Twitter: @PMP_J <https://twitter.com/PMP_J>
>>>>
>>>>
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel at lists.freedesktop.org
>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20140808/3d8f8cf2/attachment.sig>