[gst-devel] plugin opengl multiplatform

Fri May 23 21:18:48 CEST 2008

Hi,

I have made a build tree that contains the plugin source code and 7
examples  that show all the capabilities of the plugin.

-{RGB, I420, YV12, YUY2, UYVY, AYUV} -> glgraphicmaker -> {GL flow}
-{GL flow} -> glvideomaker ->  {RGB, I420, YV12, YUY2, UYVY, AYUV}
-{RGB, I420, YV12, YUY2, UYVY, AYUV} glimagesink
-GLSL and multiple FBOs
-graphic FPS can be greater (or synchornised) than the video FPS, it means
that the opengl scene can be at 2500 FPS and the input video framerate at 25
FPS.
-the opengl scene can be made in the client code (no need to compile a
specific filter)
-GLEW 1.5.0, OpenGL 2.1

It works fine on win32 on a radeon 9800, X800 and a nvidia 6600.

The sources and examples compiles on win32 and linux too, but it seems that
only RGB works on linux. When input is yuv,  the colors displayed are
like-purple. Maybe I have not using a correct configuration with fglrx
drivers, I am not really aware about this. The gstxoverlay interface works
only on win32. I prefer to resolve the color problem first.
But for now I will take a pause and I am sure you guys are more friendly
with linux developement. So you will resolve it faster than me, if you need
it.

I can give a SVN acces of the source code to the gstreamer admins who want
test it. The repository contains all what you need to compile it on linux
and win32.
Just send me an email.

Sincerely

Julien I.

http://vulvteam.free.fr/gstreamer/gstreamer_opengl.zip
http://vulvteam.free.fr/gstreamer/gstreamer_opengl_2.zip<http://vulvteam.free.fr/gstreamer/gstreamer_opengl.zip>
http://vulvteam.free.fr/gstreamer/gstreamer_opengl_3.zip<http://vulvteam.free.fr/gstreamer/gstreamer_opengl.zip>

2008/5/19 Julien Isorce <julien.isorce at gmail.com>:

> Hi,
>
> I do not know exactly. I have want to be sure that it works on linux too. I
> have tested it on a Kubuntu, it works on my 6 testcases but it does not say
> that it works.
> Then I have to clean up code etc.. and then it depends on what the
> gstreamer admins decide.
> I hope at the end of this week.
>
> Sincerely
>
> Julien
>
> 2008/5/19 KwangYul Seo <kwangyul.seo at gmail.com>:
>
> Hi,
>>
>> I wanted to use gst-plugins-gl in Windows, but I gave up because it was
>> not portable. Your work seems to be based on win32 and portable, so I am
>> looking forward to seeing your work. When are you planning to release your
>> source code publicly?
>>
>> Thanks,
>> Kwang Yul Seo
>>
>>
>> On Fri, May 16, 2008 at 9:42 AM, Julien Isorce <julien.isorce at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I have found a way to implement the gstxoverlay interface and this is a
>>> link that demonstrates it :
>>> http://vulvteam.free.fr/gstreamer/gstreamer_opengl_3.zip<http://vulvteam.free.fr/gstreamer/gstreamer_opengl.zip>is a zip file (9 MO) that contains 3 avi example files:
>>>
>>> videoxoverlay.avi: The Id of a QWidget is passed to the glimagesink
>>> through the gstxoverlay interface
>>> cubexoverlay.avi: The Id of a QGLWidget is passed to the glimagesink
>>> through the gstxoverlay interface, the video is played over a "rotating"
>>> cube
>>> mousexoverlay.avi:The Id of a QWidget is passed to the glimagesink
>>> through the gstxoverlay interface, the video is played over a "static" cube.
>>> The mouse moves the cube.
>>>
>>> I give again the previous links :
>>>  http://vulvteam.free.fr/gstreamer/gstreamer_opengl_2.zip<http://vulvteam.free.fr/gstreamer/gstreamer_opengl.zip>
>>> http://vulvteam.free.fr/gstreamer/gstreamer_opengl.zip
>>>
>>>
>>> Next step is to make a linux build in order to test this plugin on a
>>> Kunbutu for example. (it should work but I need to verify it)
>>>
>>> But for now the plugin is finished and works fine on win32.
>>>
>>> Sincerely
>>>
>>> Julien I.
>>>
>>> 2008/5/6 Julien Isorce <julien.isorce at gmail.com>:
>>>
>>> Hi,
>>>>
>>>> The third element, glvideomaker is now complete. It handles every
>>>> RGB-like. It also handles I420, YV12, YUY2, UYVY and AYUV, trough FBO and
>>>> GLSL. (GL flux to video flux). It must be connected to the glgraphicmaker
>>>> (symetric element).
>>>>
>>>> To exposes the capabilities of the plugin I am working on, especially
>>>> about colorspace conversion., I made other showcases :
>>>> http://vulvteam.free.fr/gstreamer/gstreamer_opengl_2.zip<http://vulvteam.free.fr/gstreamer/gstreamer_opengl.zip>is a zip file (15.6 MO) that contains 3 avi example files.
>>>>
>>>> It was not easy to manage multiple FBO and one FBO with multiple
>>>> attached textures, especially correclty switching between them. Then shader
>>>> programs for "from GL to video conversion"(glvideomaler) were much complex
>>>> than "from video to GL conversion"(glgraphicmaker).
>>>> But now I think it's ok.
>>>> And shader programs could be optimzed by everyone who want help, after I
>>>> share the code.
>>>>
>>>> So before to share the code I want to implement the gstxoverlay
>>>> interface. But I have to get ride of freeglut that is not an easy task.
>>>> If someone want to help me in this last task, it will be appreciated.
>>>>
>>>> Other thing, I never tested my plugin on linux but I plan to do it. I
>>>> should work thank's to freeglut, I hope :P. But I am not really aware with
>>>> automake and autoconf. Help about this would be appreciated too.
>>>>
>>>> I am sorry, when I started the plugin I would just make a "patch" for
>>>> the existing one. But the now I see that the mecanism is completely
>>>> different (some parts are similar and the existing code helped me a lot at
>>>> start). Also the existing code was less commented and it was quite
>>>> impossible to enter. But I understand that it's a project made on spare
>>>> time. Anyway, I could not start without the existing one.
>>>>
>>>> So, what the best way to proceed. Patch the existing one ? (impossible
>>>> as I said), Fusion with the existing one ? (we can try), an other plugin ?
>>>> (name?), an indepent plugin ?
>>>> I would be happy to integrate the plugin to the gstreamer repository,
>>>> but if you think (after looking the code) that the plugin is not that the
>>>> gstreamer ommunity want, I can make an independent plugin.
>>>>
>>>> Sincerely
>>>>
>>>> Julien
>>>>
>>>>
>>>> 2008/4/21 Julien Isorce <julien.isorce at gmail.com>:
>>>>
>>>>>  Hi,
>>>>>
>>>>> Some news about the plugin I am working on my spare time:
>>>>>
>>>>> The plugin uses the opengl API. It uses also glew to automatically
>>>>> manage extensions.
>>>>> For now there is two elements (implementation is finished)
>>>>> The first one is named glimagesink (as the current plugin-gl from David
>>>>> A. Schleef). It handles every RGB-like. It also handles I420, YV12, YUY2,
>>>>> UYVY and AYUV, trough GLSL (OpenGL shading language). (this is done and
>>>>> tested on an ATI 9800 pro, an ATI X800 GTO and an recent NVIDIA card (I can
>>>>> search the exact name if needed) )
>>>>> The second element is named glgraphicmaker, it handles the same video
>>>>> format inputs as the glimagesink. The output has the type video/x-raw-gl (as
>>>>> in the current plugin-gl from David A. Schleef). I prefer to name it
>>>>> "glgraphicmaker" instead of "glupload" because it is possible to define
>>>>> reshape and draw callback from a gstreamer client code (usefull when someone
>>>>> want to have a complex opengl scene without making an other filter). The
>>>>> most important thing is that the graphic FPS can be synchronised or bigger
>>>>> (I mean as fast as your graphic card can) than the input video
>>>>> framerate (the client code can to turn on or off it). The glimagesink can be
>>>>> connected to the glgraphicmaker in order to see the opengl scene.
>>>>>
>>>>> Now I am currently working on the symectric element that it is named
>>>>> "glvideomaker". I prefer this name instead of "gldownload" for the same
>>>>> reasons.
>>>>>
>>>>> The other thing that I want to talk about is the "freeglut" part in
>>>>> this plugin. I use it to create the windows easily but there is a
>>>>> limitation. It's not possible to implement the gstxoverlay interface because
>>>>> freeglut cannot receive winId from an existing window. So, there is two
>>>>> solutions to resolve this problem. The first one is to extract the small
>>>>> part I used in freeglut and just put it into the plugin. The second solution
>>>>> is to use an other opengl toolkit (glut-like) that could permit to implement
>>>>> the gstxoverlay interface.
>>>>> The second solution is possible because the deep mecanism of this
>>>>> plugin will be the same (just by replacing freeglut fucntions by the
>>>>> identicals functions from an other library)
>>>>> What solution do you expect to be the best? If it's the second one,
>>>>> which library are you thinking ?
>>>>>
>>>>> Sincerely
>>>>>
>>>>> Julien I.
>>>>>
>>>>>
>>>>>
>>>>> 2008/4/12 Julien Isorce <julien.isorce at gmail.com>:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Link : http://vulvteam.free.fr/gstreamer/gstreamer_opengl.zip is a
>>>>>> zip file (5 MO) that contains 4 avi example file.
>>>>>> The videos show capabilities of the plugin I am working on.
>>>>>>
>>>>>> It uses freeglut API but it could use an other glut-like multiplatform
>>>>>> toolkit. The mecanism should be the same. I'ill make the plugin able to use
>>>>>> any of them.
>>>>>> Some work has to be done before sharing the code.
>>>>>>
>>>>>> I could say more about the plugin capabilities but examples are
>>>>>> explicits. And I do not want overload this email.
>>>>>> I am waiting remarks, requirements, advises etc ...
>>>>>>
>>>>>> Julien I.
>>>>>>
>>>>>> ps: Thanks to C.C. for a lot of discusions and advises.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>> -------------------------------------------------------------------------
>>> This SF.net email is sponsored by: Microsoft
>>> Defy all challenges. Microsoft(R) Visual Studio 2008.
>>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20080523/afb9ba05/attachment.htm>