Questions about the EGL GLES sink

[Carlos Rafael Giani]

Hello,

I have been trying to integrate Vivante's "direct texture" extension for 
OpenGL ES into the eglglessink. A modified (actually, hacked) 
eglglessink version worked already.
There are some concerns though:

1) The only difference between direct textures and regular ones is the 
way pixels are uploaded. glTexImage2D() is replaced by glTexDirectVIV() 
or glTexDirectVIVMap(). The latter is particularly interesting, since it 
expects a physically contiguous buffer, which is what hardware decoders 
render into. The direct textures are just regular GL_TEXTURE_2D textures 
for the rest of OpenGL. Still, this requires changes to the uploading 
process and the way stride and vertical padding is handled. In 
particular, the long and elaborate logic in 
gst_eglglessink_fill_texture() cannot be used.

2) glTexDirectVIV() and glTexDirectVIVMap() accept YUV formats directly, 
so no conversion with shaders is needed. This also means that the 
texture does not have to be attached to multiple texture units for the 
conversion - just one glActiveTexture() call is enough. I could always 
use the glTexDirectVIV* calls with the GL_RGB(A) formats and do the 
conversions in the shaders, but that doesn't seem efficient to me if the 
GPU can do it directly.

3) As a consequence of (2), the sink caps are different. Plus, the COPY 
shader has to be used for various YUV formats, because there would be 
duplicate conversions otherwise (by the GPU directly and then by the 
shader).

I could change things so the COPY shader is used for all directly 
supported formats, and for the other formats, use glTexDirectVIV* calls 
with the GL_RGB(A) format set, and do the conversion in the shader. But: 
for example, the i.MX6 has another coprocessor - the IPU - which can do 
colorspace conversions in hardware, very quickly.

So, what are the options? I see two: stuff all of this into eglglessink, 
or write a separate sink. I am not sure what is better. The changes to 
eglglessink seem substantial to me. Then there is the fact that 
eglglessink differs wildly between 1.0.x and 1.1.x , and eglglessink 
will eventually become a glimagesink , so this thing is in a strong flux 
in 1.1.x . (Also, if I use the sink from 1.1.x, I need to pull in 
several other dependencies from 1.1.x; this might not be a problem once 
1.2 comes out, because then I can name 1.2 as a stable dependency.)

So, opinions? Thoughts?

Carlos