[compiz] [ANNOUNCE] Compiz feature branch compiz++

[Dennis Kasprzyk]

Kristian Lyngstol wrote:

> On Wed, Dec 24, 2008 at 01:48:17PM +0100, Dennis Kasprzyk wrote:
>> - No direct access to member variables: Everything is now done with
>> getter and setter functions. This helps with the problem where every
>> variable addition broke the plugin ABI. While this usually is not a
>> problem in the development tree, we can not do it in a stable tree.  Even
>> if it is needed to fix a bug.  It also gives us more control what other
>> plugins can do, so that broken plugins can not mess up our core
>> structures.
> 
> How does this relate to the fundamental issue where we simply have way too
> many variables available under one or two data structures? I believe the
> biggest problem we've had is the sheer size of the core structures and
> their lack of documentation which has led to misuse. The get/set for ABI
> compatibility is not unimportant, but it doesn't solve the underlying
> problem.
> 

The separation of the compositing and opengl rendering  into individual plugins, creates already smaller data structures. With the drop of the multi-display and multi-screen support, also all the X Atoms and GL function pointers/properties have been moved out into namespaces, and are not part of the data structures anymore.
The use of setter/getter functions also helps us to reduce API here, because I've only added this functions for variables, that are currently needed in other plugins (there might still be some functions missing for other plugins).
This doesn't fix the problem of missing documentation, but at provides a cleaner API.

>> - Composite/OpenGL seperation: All XComposite handling is moved into the
>> composite plugin and the opengl rendering into the opengl plugin. This
>> allows compiz to run also as normal window manager without compositing
>> (still needs some work and a few new plugins), but also the creation of
>> other rendering backends (XRender, clutter, ...).
> 
> What are the specifics for this interface?
>
The composite plugin only provides 3 base functions preparePaint, paint and donePaint (same functionality like preparePaintScreen, paintScreen, donePaintScreen in C compiz)
for normal plugins. For rendering plugins like the opengl one it has an additional prepareDrawing and paintOutputs function interface. PrepareDrawing is called before any rendering starts and paintOutputs is called in the core implementation of the screen "paint" function chain.
This interface should be flexible enough to support any rendering system, even scene graph APIs.
The opengl plugin then implements all the other drawing functions that are currently in the C version of compiz.

>> - Tiled textures: Modifications to the texture system allow to have more
>> than one texture per pixmap and also to have other plugins that provide
>> texture from pixmap functionality. The new copytex uses the (slow) "copy
>> pixmap content to texture" approach to provide texture from pixmap
>> functionality. It's main advantage is that it supports pixmaps bigger
>> than the maximum texture size. In this cases a pixmap is split into
>> multiple textures. Compiz will use the glx texture-from-pixmap extension
>> for pixmaps/windows but will fall back to the copytex plugin if thex are
>> bigger than the maximum texture size.
> 
> What are the specifics for this interface?
> 
The opengl plugin allows other plugins to hook into the pixmap binding system (GLTexture::BindPixmapHandle registerBindPixmap (GLTexture::BindPixmapProc);). All pixmap bind and image load (not finished yet) functions now return a list instead of a single texture. Each of the returned textures can represent a part of the original pixmap/image. In most cases only one texture will be returned.
C++ make here the whole system a little smarter. Copying the the texture list increments automatically the reference counters of the textures and deleting the list automatically frees all the texture objects. The GLTexure class is only a base class for the different Texture types (TfpTexture, CopyTexture) and avoids a complex wrapping system / a lot of function pointers, that would be in a C implementation.

>> - Reparented decorations: Like other window managers, compiz now supports
>> reparented window decorations. This will allow compiz to run as a normal
>> window manager and to be able to have decorations for windows that are
>> bigger than the maximum texture size (copytex plugin). (Currently only
>> implemented in the kde4-window-decorator)
> 
> What specific bugs does this solve, what are the concerns that it will
> bring? What is the reason for the original design? Pros and cons.
> 

Reparenting is the only way to get window decorations in sync with the window during movement in a non composited environment. I've tried different ways for window decorations in the last 2 years, but reparenting is the only really working system.

> Is this an optional feature, or is it always enabled in your branch? Is it
> implemented as a plugin?
> 
No really. The normal (current C compiz) decoration system is still the same, but the invisible input window is now reparented with the client window. This was the only way to provide a consistent handling of the window frame, without the need to provide two different paths (one for reparented frames and one for a non reparented input window).

>> - Dropped multi display and multi screen support *: The multi display
>> support is not completed, and the multi screen support is almost
>> unmaintainded. Additionally, our normal proposal for bugs in the multi
>> screen support is to start one compiz instance per screen. Dropping
>> multi-screen support has the additional benifits of:
>> -- Per screen plugin lists (e.g. cube on one screen and wall on the
>> other) -- Rendering of one screen can not block the rendering of the
>> other screen -- Different libGL per screen (with LD_PRELOAD)
>> -- Simplier plugins
>> -- Simplier option handling
> 
> I believe this has been a separate discussion, though it seems it never
> came up on any of the lists(?). I believe a proper discussion should be
> had (I could've sworn we had one already), but I suspect this could be
> implemented on master without too much grief.
> 
It can be implemented in master, but it will introduce a huge changes to all plugins, so that all plugins will need to be ported anyways.

>> - New plugin interface: The compiz WRAP/UNWRAP interface is perhaps the
>> most efficient system to create a plugin funcion "call chain", but it is
>> not the the best for compiz. In compiz we have a lot of plugins loaded
>> that hook into the drawing functions, but do nothing most of the time.
>> Only when activated (keybinding/dbus/event) do they usually draw
>> something. The end result is that most functions get called, then the
>> plugin checks if it is active. The plugin
>> then calls the function of the next plugin in the call chain.  The
>> following plugin repeats the cycle. I've measured that we are loosing 8 -
>> 15 % of CPU time here. The new system allows plugins to dynamically
>> disable "wrapped" functions if they are not needed. The core then only
>> needs to check some boolean variables, and will only call the functions
>> that are really in use. I've kept the word "wrapping" in the system so
>> that every compiz developer knows what it does, even if it does not fit
>> the new system well.
> 
> How exactly does this new system work? This represents a major change for
> Compiz, and far more important than performance is the maintainability
> over time. How do you envision this new system handling in the development
> cycle?
> 
The system is almost the same like the current one from the plugin point of view. Plugins can simply ignore the ability to disable the wrapping of different functions in the first place. Once the plugin is really working, it can then be optimized, to use functions only if they are really needed. On the implementations side, I've implemented a lot of macros to make it easy to add new "wrapable" functions to the core/plugin plugins. I will write a detailed documentation about the system as fast as I can.

> When are these 8-15% of CPU time? Is it during a time when Compiz is
> heavily into damaging (ie: during animation) or a similarly intense
> situation, thus the 8-15% of CPU time is actually 8-15% of the CPU, or is
> it measured when Compiz is otherwise idle, and the 8-15% is 8-15% of ~0?
> 
This 8-15% are results of normal profiling of compiz.

> Generally speaking, my Compiz uses roughly no CPU, and if it's 8-15% of
> nothing, I hardly think it's worth the gain, which is why this matters.
> But depending on the implementation, this can probably be done pretty
> cleanly without too much tinkering with existing plugins.
> 
While the 8-15% are not a problem during normal usage, they can give you the 2 missing FPS to make compiz still feel smooth under heavy load.

>> - CMake build system: Everyone is happy if the build system (autotools)
>> works, but starts to cry if it doesn't. At least that is my impression
>> after more than 2 years of compiz development. In compiz fusion we have
>> already decided to switch to cmake after the next stable release (0.8),
>> and we also provided cmake tarballs for a while. CMake provides a real
>> great documentation, and in my opinion it is much easier than autotools.
> 
> This probably should be done directly on master after 0.8, but I'm still
> missing some specifics. So far, all I've heard about CMake have been
> praise, and I find it hard to believe entirely in a product when all I
> hear is good stuff. There's always a con... Does anyone care to enlighten
> me and the list?
> 
>> - Port to C++: With C we have to write almost the same code in each
>> plugin and run into the same bugs over and over again. A lot of this can
>> be avoided with C++. C++ allows us also to do more and this in a easier
>> way: -- Smarter function callbacks with Boost
>> (boost::function/boost::bind) -- Easier and smarter privates system (get
>> the plugin struct for a given core struct/old FOO_SCREEN (s), FOO_WINDOW
>> (w) macros). The new system hides most of the ugly handling from a plugin
>> developer and provides a new simple and ABI safe way to work with plugin
>> plugins (plugins that expose features/functionality to other plugins)
>> -- Constructors/Destructors allow easier initialisation/cleanup for a lot
>> of systems.
>> -- Containers avoid the implementation of lists and resizeable arrays
>> over and over again.
>> -- Containers like maps and other smart classes can improve performance
>> in several areas.
>> -- ...
> 
> I'm sceptical to switching to C++ _and_ implementing all these changes at
> the same time. Specially the way it's happened. I'm not entirely sure C++
> will solve all the real problems either (like the biggest problem which is
> lack of good documentation). For now, I'd like to discuss the individual
> changes suggested, or presented, I should say. Before I'm willing to
> discuss the move to C++.

C++ doesn't fix all the problems, but it simplifies the writing of new plugins and avoids a lot of code duplication. 

> 
>> This branch is my proposal for a possible compiz future. The decision to
>> make this the future of compiz is something that everyone involved in
>> compiz should make. Espesially because it would need a huge amount of
>> work to get all plugins ported. In my opinion, it would be better to stay
>> with the old system if such change would create a fork again.
> 
> I'm really alarmed with how this branch/project evolved, so I'm sceptical.
> Essentially 6 months of development has been poured into a project we're
> not supposed to talk about that changes more or less everything that is
> compiz, and there's been no open discussion along the way. This has been
> an ongoing problem with Compiz for as long as I've been involved, and it
> has, in fact, been the single biggest challenge in working with Compiz. I
> regret to see that you are carrying this tradition on Dennis...
>
It's not like there have been no discussion about the topic. All major developers have been informed, and I wouldn't have continued to work on all the stuff without positive feedback.
I've kept the work secret, to simply provide a code that is in a usable state. It took longer than I've expected and my plan was to have all core plugins ported already before making it public.
 
> I am in no position to reject or accept such a huge proposal, but I'm very
> curious as to how to maintain it. Essentially we're turning everything
> into a 0.2 version again, since we'll have to learn Compiz all over again.
> 

The main point here is, that it's better to move back once and do all big changes at once, instead of doing it for every feature over and over again. I wouldn't say we move back to 0.2, we have a lot not ported plugins, but the core functionality is there, and from my point of view there are no big stability issues.

> I know I might have uttered differently before, but I've come to realize
> that you do not solve the fundamental problems of a program just by
> starting from scratch, and this is close to that.
> 
If you really look at the interfaces, you should see that everything is almost the same. Some parts are moved out to different plugins, other got merged, and you have to use getter/setter functions instead of direct variable access. The functionality how the WM core and the rendering works is still the same.

> From my point of view, it is much better to present each of these changes
> as individual ideas, work over the idea, then start looking at an
> implementation and patches. I see a lot of good ideas, but I don't see any
> discussion with regards to the design or implementation. This will require
> much work in the area of documentation and communication before I, at
> least, am willing to give this a thumbs up (to the degree my thumb matters
> in this situation).
> 
If you implement the features individually you will have to change more than 50 plugins over and over again. And we don't have the manpower to do it.

> For the record. I still agree with many of the problems you put forward,
> but I believe that the biggest challenge in Compiz for the future lays not
> in the code, but in the way we work, and in changing compiz from a
> research project to a throughly professional project. That, and the lack
> of time.
> 
> - Kristian