GLX redirection extension

[Felix Bellaby]

On Thu, 2006-07-20 at 13:57 -0400, Kristian Høgsberg wrote:
> On 7/20/06, Felix Bellaby <felix at bellaby.plus.com> wrote:
> > I think that I may have found a potentially useful way out of some
> > problems within the current compositing framework. Basically, the server
> > has information that the compositor would find useful.
> 
> I'm not sure what problems you're trying to solve here, but we don't
> need extra protocol for redirecting GLX.  When a window is redirected
> with COMPOSITE, all rendering to that window is supposed to go to an
> offscreen pixmap.

That is the principle problem. Rendering GL directly onto the screen is
much faster than rendering into a pixmap and copying the pixmap onto the
screen, especially if the second operation is performed using a texture
map. The current situation allows you to use a composited desktop with
2D applications _OR_ use GL for serious applications like 3D modeling.
You can not do both at the same time.

With GLX redirection, the compositor could allocate areas of the screen
for use by GL applications and they could run at full speed unhampered
by the draw -> pixmap -> texture -> screen set up. This is how direct
GLX has worked under X in the past, and I can not see how this technique
can be used in the future unless the compositor treats different apps in
different ways based on their GLX requests. 

Furthermore, rendering into an off screen pixmap currently fails to take
full advantage of the capabilities of GL hardware. If the compositor /
server responded to CreateWindow requests by offering a framebuffer
object using a texture as the renderbuffer then the drawing could go
directly into the texture with no intermediate pixmap. SwapBuffer
requests on the GLXWindow drawable could move the rendering into a
different texture. This would probably provide the fastest possible
implementation of texture based compositing available on existing
hardware, and it could be done using redirected GLX in combination with
existing GLX/GL drivers. I am not convinced that patching the drivers
will achieve the same speeds using texture_from_pixmap.

Sharing Pixmaps between the applications doing their drawing and the
compositor drawing the screen can not be trivial when the applications
and compositor occupy separate processes. Telling the compositor what is
going on after the event using Damage reports works to a point, but the
GLX specs are deliberately vague about what happens when you share
drawables between uncoordinated processes. Perfectly compliant
implementations are free to respond in undefined ways, and may do so in
order to milk a bit more speed. 

For example, the current nvidia drivers crash when GLXPixmaps are used
with Pixmaps that COMPOSITE has already discarded following the
unmap/resize of a window. nvidia have undertaken to fix this, but I do
not think that it is strictly a bug. The nvidia drivers also seem to
freeze GL applications when a compositor tries to access the COMPOSITE
Pixmaps directly using GL. Again, this may be a bug that they can fix,
but it may have a performance cost.

We seem set on the hope that every single GL driver developer will be
able to solve these kinds of problems on behalf of the compositor,
without losing performance or expecting any thanks. I think that it
might be more sensible and efficient to design the compositor to do
things using GLX spec compliant techniques, rather than leaving it
largely to others to try to fix things up behind the scenes in their
drivers.

>   The two big issues with COMPOSITE today is that
> accelerated OpenGL (direct or indirect) and Xv doesn't respect this
> and render directly to the front buffer.

The latest nvidia drivers do perform accelerated GL drawing into off
screen pixmaps when compositing is enabled. However, this is slower than
doing it directly onto the screen back buffer. Xv renders directly onto
a screen overlay because this is the only way to render quality video at
acceptable framerates.

>   Extra protocol or API won't
> solve this, we "just" need to fix the implementation to consistently
> redirect all kinds of rendering for a redirected window.

Forcing all direct OpenGL and Xv rendering into Pixmaps might make them
obey the dictates of the current COMPOSITE regime, but it might also
defeat their entire purpose - speed.

GLX redirection would enable the COMPOSITE regime to loosen its shackles
so that maximum speeds could still be obtained. There might be other
ways of doing this that are more immediately palatable, but I think that
we need to address the problem of allowing existing software and
hardware to work at the speeds that they currently enjoy.

> There is one case where we will need extra protocol (this will be an
> extension to the XF86DRI protocol): when doing direct OpenGL rendering
> to a redirected window.  In this case all the rendering happens in the
> client without the X server knowing, but the compositing manager will
> need damage events for the pixmap in order to be able to recomposite
> it as changes happen.  For this, we need an XF86DRI request that
> allows libGL (the direct rendering client) to post damage on a window.
>  But note, none of this will be visible in the API.

I would imagine that DRI-compatible libGL will use this DRI request to
inform the server of damage while processing a SwapBuffer call. nvidia
appear to have followed a similar approach in their drivers.

GLX redirection would only be visible within the compositor, leaving the
API used by ordinary applications unchanged. The compositor will have to
be aware that some applications want to draw directly into the screen
framebuffer in order to allow them to do so, and I think that an API
change is needed to achieve this aim. 

Felix