[Mesa-dev] KWin and Mesa

[Lucas Stach]

Hello Martin,

sorry for the kick-in, but I feel obligated to speak up here.

First of all you wrote a rather long text, but you don't make a single
point _how_ to make the collaboration between mesa and kwin devs any
better. It seems you just want to explain why you are doing things the
way you are doing them.

Please understand that mesa is also a community driven project and many
of the contributions (especially for hardware drivers) come from
volunteers, too. What you are calling for is to make them watch for
regressions in all applications using mesa. I don't see why those devs
should have more time for such things than kwin devs. It just won't work
out to let one side do all the work. Trying to setup an automated
regression test together with the kwin devs sounds more like a plan. It
would end up in some work for both sides, but in the end it will result
in a better quality.

Also as said half a year ago, blacklisting features in kwin based on
some renderer string is a _very_ bad idea. OpenGL has ways standardized
ways to query functionality. If advertised features do not work as
expected this is a bug in mesa and should be reported, not silenced on
the applications side. Writing and testing applications only against the
proprietary drivers doesn't make the situation any better as these
drivers let you get away with a pile of brainfuck, which you won't find
in any OpenGL standard. I think in this respect kwin devs (as well as
every other app developer) should also ask themselfes if their stuff is
really valid OpenGL. It's not always mesa breaking stuff, sometimes it
is also the application breaking mesa.

You should really try to find a way which both sides can agree with.
Demanding one-sided things is not the way to go. We can't take the
hassle of testing your application from you. You will find many open
minded people in the mesa community who will gladly help you if
something breaks, but there is also some work you have to do. Real
communication between devs of both sides is the way to go if you want to
make for a better user experience.

Regards,
Lucas


Am Dienstag, den 19.04.2011, 16:52 +0200 schrieb Martin Gräßlin:
> Hi Mesa-devs,
> 
> yesterday I published a rant about Mesa breaking KWin and given some 
> comments on Phoronix Forums it seems like there is the wish for more 
> communication between our development groups and so I want to start it. Please 
> note that I am not subscribed to this mailing list, so please keep me in CC (I 
> might not be able to reply this week at all). It is my wish to never have to 
> rant about the state of Linux drivers any more and that I never have to see 
> Mesa breaking KWin again.
> 
> First of all I want to give a little bit personal history to help you 
> understand why I so far did not contact you and up to now wrote two rants 
> about Mesa breaking KWin. Let's go back to 2010 and the 4.5 release. In March 
> 2010 I finished my Master Thesis and in April 2010 I started my first job. KDE 
> hat the feature freeze for the 4.5 release around End of April, beginning of 
> May. All new functionality including the Blur and Lanczos filter were 
> implemented at that time. Given the change in my life due to end of my studies 
> I did not contribute much to the release.
> 
> At the time when we implemented the features the current Mesa version was 7.7. 
> Version 7.8 was under development and when it got released marked as a 
> "development version". I read this information and considered "ok we don't 
> have to deal with 7.8 - it is development". At that time I had a notebook with 
> NVIDIA graphics and an old system with a rather modern Ati card, with a 
> crashing X server if I tried to use the radeon driver. I had no chance to test 
> Mesa drivers at that time!
> 
> In the time leading to the 4.5 release KWin had no active maintainer. Lubos 
> had been inactive for quite some time and made me maintainer in November. At 
> that time I considered Lubos still to be the maintainer and to be responsible 
> for decisions whether to ship the new features or not. I considered myself 
> only responsible for my own code (which did not cause any problems in that 
> release). I was also running the stable version of KDE (4.4) at that time and 
> the development only for testing.
> 
> During the beta phase we realized that we had a problem. Mesa 7.8.2 was 
> marked as stable (which I did not expect due to the fact that 7.8.0 and 7.8.1 
> were unstable) and distributions started to include it. Users were complaining 
> about broken features mostly concerning blur and lanczos filters and mostly 
> with Ati and Intel drivers. Nobody in our development team had an Intel system 
> at that time. I had had access to a system before through a friend of mine, 
> but unfortunately it broke down in exactly that important time. Later on my 
> friend got a new Intel powered device but run Debian Testing on it which did 
> not show any of the problems the users reported with Mesa 7.7. Concerning Ati 
> I knew that Fredrik had been in contact with Mesa developers and that all the 
> new functionality had been implemented on his Ati systems. So we knew that the 
> functionality worked at least with some systems.
> 
> With the looming release and more and more obvious problems we faced two 
> possible solutions: remove the code completely (disable by default would not 
> have solved it) or try to get it working somehow. I did not see any reason why 
> we should have punished the users of working drivers (e.g. NVIDIA) because 
> other drivers did not work. After all the new functionality was an important 
> feature for our provided user experience and our designers and the Plasma team 
> were demanding it. So we had only the option to make it work. Now you can 
> imagine how difficult it is to workaround bugs in hardware you do not have. My 
> solution was to implement a black list and to crowd source the creation to all 
> our users. Another of the changes was to have the test whether to use direct 
> rendering in an external application (due to drivers crashing when trying to 
> create a GLContext) and there the "hack" was introduced which now backfired.
> 
> Now why did I not contact you when we were facing these problems? Given that I 
> had a day job, 2 h travel each day and trying at the same time to make the 
> experience as smooth as possible for our users in the evenings, I seriously 
> had not the time to even think about it. And what would it have changed? We 
> need to get KDE supporting the drivers out there and not the next version! I 
> am sorry that I did not contact you at that time but I think everybody will 
> understand that sometimes as a volunteer developer you don't have the time to 
> do everything which would help. Last but not least I did not feel responsible 
> as I was not the maintainer of KWin. 
> 
> When we released 4.5 I was positive that we had successfully established a 
> black list which ensures that no user would face issues. Unfortunately I was 
> wrong. Users still faced the issue and even worse: the desktop started to 
> freeze if you changed settings in KWin. A problem which had been completely 
> unknown to us before the release (there was a bug report by one user but it 
> was unconfirmed). None of us was able to reproduce because we did not have 
> the hardware. At the time when finally a user was able to investigate it, he 
> discovered that it was a bug introduced in Mesa 7.8 and not present in the 
> development version of Mesa 7.9 - one of the reasons why Maverick shipped a 
> development version of Mesa 7.9.
> 
> Due to the bugs KWin had hit in Mesa, KDE got a pretty bad Media coverage. 
> This was something I did not like. The bugs were not in our software. I have 
> no problems with admitting bugs in my part and I take blame for it, but I 
> cannot stand being blamed for problems due to 3rd party breakage. So I wrote a 
> blog post explaining the situation. Explaining the steps we did to ensure that 
> there won't be problems and I did not want to have it as a rant, though it 
> seems to have been read as such by some people. I really did not want to piss 
> you off, I just had to explain that the bugs are not in KDE, but in a 
> different layer of the stack and that to a certain degree we did not even know 
> about the bugs prior to the release.
> 
> Now let's fast forward to last week and why I again wrote a rant (and this 
> time it was intended as a rant) without contacting you. Again it is quite 
> human: I was informed about the change which broke KWin last week. Last week 
> I was unfortunately ill (I am still fighting the last parts of my cold) and 
> nevertheless went to work and was in the evenings so exhausted that I most of 
> the days just dropped to bed. On Saturday I flew to New York for a business 
> trip and just now I am spending my evening sitting in a hotel room writing 
> this mail. Seriously I just did not have the time to contact you.
> 
> So why a rant? Seriously: I am disappointed. I am very, very sad about what 
> happened here. After all the buzz about my blog post more than half a year ago 
> it happened again that Mesa developers broke our software. I think I cannot 
> describe in words how I feel about it. Now the situation has changed and I am 
> maintainer of KWin and I do feel responsible for the complete experience. I 
> hope this somewhat explains why I did not contact you prior to writing the 
> blog post. It is important to know that I also wanted to get the right people 
> to read it: that is the Kubuntu devs to find a solution ASAP to not have a 
> broken release. Kubuntu is one of our most important downstreams and I do 
> care that they have a great release. Given the combination of illness and 
> business trip I had not seen any other way than to write a blog post. Sorry 
> that I put my frustration into it. This is unprofessional, but also part of my 
> character. You know we Germans are direct and I speak the truth even if it 
> hurts. Now feel free to shoot the messenger of bad news, but sorry it is the 
> truth that Mesa just broke KWin.
> 
> So this is already rather long and should be enough for the history part. 
> Let's now look on how to improve the situation. Therefore I try to explain how 
> I see this. Mesa is our most important upstream and KWin (together with Compiz 
> and Mutter) are your most important downstreams. I am sometimes not sure that 
> you are aware of this. KWin is probably the OpenGL compositor with the largest 
> user base or the second largest after Compiz. You should be aware that there 
> is nothing worse than a broken compositor. It doesn't matter if you now 
> support game foo or game bar if the compositor breaks. Please keep this in 
> mind. Given how important KWin is for you as a downstream I am really 
> surprised to see that you do not do regression tests against KWin. What can we 
> do to help you there?
> 
> Please understand that KWin has only three regular developers. We are all 
> volunteers and are doing development in our spare time. Please understand that 
> I personally do neither have the time nor the patience to follow the upstream 
> development. In a perfect world I would do KWin development as my day job and 
> would have the time to follow the upstream development. In reality I have to 
> choose to spend my time on developing KWin or follow upstream development - I 
> don't have time for both. Please also understand that I am not able to test 
> your drivers. On my notebook I was using nouveau for some time, but reducing 
> the battery life by more than an hour and not being able to suspend is a high 
> price for running free drivers. On my primary system I tried to use the radeon 
> driver. I even tried to compile it from sources. After not being able to 
> succeed after spending half a weekend I am waiting for new drivers to hit 
> Debian testing. Please understand that I am not willing to switch the 
> distribution just to get newer drivers. This is my main work system and for 
> development I want to use a distribution I know and trust. Yes Kubuntu might 
> be an obvious choice but I am unhappy with parts of the underlying Ubuntu 
> stack. On my third system which actually has an Intel GPU I am not able to 
> install development drivers as it is a company notebook. I am glad that I can 
> use the latest version of OpenSUSE so that I have a decent combination of KDE 
> and Mesa.
> 
> Anyway even if I were able to follow the upstream development it would not 
> help much. The combinations of hardware, driver, mesa, kernel and distribution 
> are way too high that our team can ensure that it does not break. This is 
> something only you can ensure. I don't know if you are aware of it, but our 
> latest version of KWin is currently used by users from Mesa 7.7 to Mesa 7.11. 
> Our users are hitting bugs you fixed more than a year ago and we cannot just 
> ignore our users and tell them to compile the latest Mesa version. Yes there 
> are hacks in KWin - one of the hacks backfired last week, but this is the 
> reason why we have hacks. We need to support yesterdays, todays and 
> tomorrows Mesa versions. If we needed a hack to workaround an issue in an old 
> version or KWin would crash, nobody is helped if we contact you and you tell 
> us that the bug is fixed. Please understand that our users have the drivers 
> the distributions give them, it is not under our control.
> 
> We have developed a very complex system to ensure that our users get the best 
> possible experience. Due to the problems we had with 4.5 and Mesa announcing 
> support for features not supported by the chip (consider emulating GLSL in 
> software - great for games deadly for compositors) Fredrik developed an 
> elaborated system to detect what chipset is installed in the system and 
> enabling features based on what the chipset really supports instead of 
> listening to what Mesa tells us. As far as I understood you recently changed 
> the way how the renderer and version string looks like. This means that you 
> completely broke that system. After all that had been said about the 4.5 
> release I was sure that you know what KWin reads from your announced strings. 
> Now it might be that we should not parse this information and of course you 
> are right with saying so, but you know what Linus would answer? "Never break 
> your userspace!" This is what just is happening. Now we are lucky and there is 
> time to adjust our code base till our next major release, but again we need to 
> at least support both Mesa 7.10 and 7.11 in one code base. Kubuntu and 
> OpenSUSE users (both using Mesa 7.10) will upgrade to 4.7 as soon as it will 
> be released and Kubuntu 11.10 will be 4.7 with Mesa 7.10. I think you get the 
> problem we are in. We cannot just remove the detection code due to the 
> experience we had with 4.5 and due to the fact that we want to give the users 
> a good experience. If the hardware does not support shaders the CPU is most 
> likely not powerful enough to do useful emulation. Assumptions which are 
> correct for a game are completely wrong for a compositor. The compositor may 
> not cause high CPU usage, the compositor may not block the system. It is a 
> background service which just has to work and should never be visible to the 
> user. If the user sees KWin in top something is badly wrong. So we have no 
> other choice than to parse the renderer information as we cannot rely on the 
> extension announced. The ideal world is one thing, but the reality looks 
> different.
> 
> I hope you better understand now my position and I hope we can both work on 
> better collaboration in future. I still have the plan to submit our complete 
> compositor and all effects as a set of piglit tests but we still need some 
> refactoring and I don't know whether it is possible to take 60 kSLOC C++ code 
> as a piglit test. Please try to get your prioritize right. It is great if the 
> wine developers have the time and manpower to work together with you, but 
> please also think about that others might not have the time. Please consider 
> if it is more important to keep compatibility for an important downstream than 
> to support some more games. Think about what you can do to ensure that I never 
> have to notice about important changes in Upstream Mesa from upset 
> bugreports. The only thing what we need is stability: the drivers should not 
> break KWin. That's all I'm asking for.
> 
> Let's work together on a better composited experience on X11.
> 
> Cheers
> Martin Gräßlin
> KWin Maintainer
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/mesa-dev
>