[compiz] [PATCH] Transparent cube

[Roi Cohen]

Hi,
Recently i have worked on re-writing beryl's transparent cube, and
ported 3d plugin to compiz.
I'm attaching a patchset here that includes the transparent cube
patches (i'll post the 3d plugin when i fix some problems that didn't
happen in beryl).
Patching order: btf-ftb.patch, clip-planes.patch, plugin-events.patch,
cube-paint-order.patch, transparent-cube.patch.

Special thanks to Dennis Kasprzyk (who originally wrote some of these
patches to compiz, and helped me now with various things), Quinn Storm
(who helped me with the options), and Danny Baumann (who did most of
the events stuff).

This is what my patch includes:

Core Changes:

- BTF / FTB masks.
FTB mask allows compiz to paint the windows in reversed order, front
to back. This is needed when we paint a a cube face which is reversed.
BTF mask alone doesn't do anything, but when combined with FTB mask,
core will first paint the screen with regular order and after that in
reversed order; the plugin that set BTF and FTB mask together is
responsible for preventing core from painting the window twice. This
allows plugins to make more complicated drawing orders. This feature
isn't needed for transparent cube, but it is needed for 3d plugin.
This patch was originally written by Dennis Kasprzyk for beryl (who
also gave the idea), and now re-written for compiz by me.

-  Wrappable SetClippingPlanes
Just a new wrappable function for setting clipping planes. This is
needed for correct clipping of the cube together with 3d plugin.
This patch was written by Dennis Kasprzyk again, and ported to compiz.

Plugin changes:

- Some new events code
Specifically: Rotate sending event to cube / 3d to notify about
rotations. This is needed for 3d to initiate, and for setting
transparent cube on rotate only. Also, 3d sending an event to cube to
request to paint all viewports, this is needed for handling some
disappearing windows in 3d (the cube face becomes invisible, but the
windows that come out 3d are still visible in small parts, but
disappear if cube doesn't paint that viewport).
Last, cube is sending unfold event to 3d, needed for preventing 3d to
transform the windows in z axis when a window is between 2 viewports.

- Transparent cube
The calculations / painting code done by cube are now changed to make
cube to paint the faces in correct drawing order, back faces first,
front faces last.
Of course, i disable culling and set the desktop-window opacity to
actually make the cube transparent, and make sure cube draws all
viewports when the cube is transparent (when the cube isn't
transparent, only the front faces are painted).


This screenshot of beryl shows what happens with 3d and a window that
is on 2 different viewports. It shows why we need to change the
clipping planes for cube. Also, it shows when setting both FTB and BTF
is needed (notice how the 3 top-most windows of the right viewport
should be painted FTB, and the 2 bottom-most windows should be painted
BTF).
http://img454.imageshack.us/my.php?image=screenshotgy4.png
Another screeshot of the transparent-cube with 3d plugin (in beryl
again): http://img74.imageshack.us/my.php?image=cube3dyi6.jpg


There are some things that I'm not really sure if they work fine with
my patch, or they have bugs:

- Currently, i'm painting all the faces of the cube when using inside
cube (only when transparent), while some faces are never visible.
This is needed when zooming out the cube, so i think this is a good
idea anyway, but i'd like to know what you think about it.

- Unredirecting fullscreen windows is done by paintScreenRegion which
finds the top-fullscreen window. When doing FTB drawing, it will find
the bottom-fullscreen window, and when doing both BTF / FTB drawing,
it'll find the top and the bottom fullscreen windows. I'm not sure
weather this is really wrong, so again I'd like to hear your opinion
if my patch makes bugs on this area.

- The cubeCheckFTB function (checks if the next-painted face should be
drawn with FTB flag) seems to not work in some occasions with inside
cube (works perfectly with outside cube). I don't know why this
happens though.

- Because of the calculations-changes i made for making transparent
cube to work, xRotate has different values now from what it used to
have (it has the rotation of the top-most face, which means between
-45 and 45 for 4 sided cube, between -30 and 30 for 3 sided cube etc),
and this change results in some weird bugs when rotating the cube with
vertical rotations.
I looked at applyScreenTransform, and it looks like the rotation axis
that is used for vertical rotating expects the old values of xRotate.
This should be fixed, but i'm not sure what is the correct way to do
vertical rotations with the "new" values. Do you have any idea how to
properly do that?

- Unfold cube doesn't work any more at all. It got broken by my new
calculations, and i couldn't find any way to fix it. Could anyone of
you help in fixing it?

Any help is really appreciated :-)

Thanks,
Roi.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: btf-ftb.patch
Type: text/x-patch
Size: 4648 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/compiz/attachments/20070412/41b1fd52/btf-ftb-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: clip-planes.patch
Type: text/x-patch
Size: 6594 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/compiz/attachments/20070412/41b1fd52/clip-planes-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: plugin-events.patch
Type: text/x-patch
Size: 5726 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/compiz/attachments/20070412/41b1fd52/plugin-events-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cube-paint-order.patch
Type: text/x-patch
Size: 14345 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/compiz/attachments/20070412/41b1fd52/cube-paint-order-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: transparent-cube.patch
Type: text/x-patch
Size: 12205 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/compiz/attachments/20070412/41b1fd52/transparent-cube-0001.bin