[Libreoffice-qa] Regressions in Open Source projects ...

[Markus Mohrhard]

Hey all,

I did not want to answer nut since you chose mainly my commits here is
a short reply.

> On Fri, Mar 16, 2012 at 07:21:15AM +0100, Rainer Bielefeld wrote:
>> a) Regression "Bug 47096 EDITING: Alt + Drag and Drop for complete
>> rows impossible in particular documents" frequently forces me to use
>> 3.4.5.
>
> I had a short look at this one as a exemplary post-mortem (the bug seems to be
> assumed fixed from the developer side). What we have here is a bug that was
> introduced on the release branch (as it wasnt in the beta), which is not good.
> But such things can and will happen, the question is how to handle them
> quickly. So what can we do better here?
>
> A casual (actually I have no idea about this code domain) digging through the
> history shows that this regression was likely introduced by:
>
>  http://cgit.freedesktop.org/libreoffice/core/commit/?id=a00c5917a8deb465cc1322e140000a81340f9d69

The correct commit is
http://cgit.freedesktop.org/libreoffice/core/commit/?h=libreoffice-3-5-0&id=49a1737b7d3d61304e749c6c164165b8bf68790e

While is not good that this commit introduced a new regression it was
fixing several problems and I was testing copy/paste (the only know
bug problem area at this point) and did not find any problems.

>
> because that last touched related code areas (so this was neither adding nor
> removing a regression, but exchanging one for another). So how can we improve
> detecting such things on the release branch?

No it is not that easy. This commit fixed several crashes and problems
in the copy/paste code. We will from time to time get such regressions
since it is nearly impossible to find all affected code paths in our
highly coupled code. This has been introduced somewhere in one of the
betas and has not been marked as serious for a long time.

>
>  git log libreoffice-3.5.0.2..libreoffice-3-5-1
>
> tells me that there have been 213 commits between 3.5.0 final and 3.5.1 final, which suggest a rather limited scope of areas touched. IMHO it would make sense to focus manual testing for the touched areas of code. Doing a:
>
> for d in `find . -maxdepth 1 -mindepth 1 -type d`; do echo `git log --pretty=oneline libreoffice-3.5.0.2..HEAD $d|wc -l` $d;done|sort -n|tail
>
> shows the following stats:
>
> 5 ./svx (GUI)
> 7 ./officecfg (configuration/settings)
> 7 ./oox (docx filters etc.)
> 7 ./writerfilter (Writer)
> 9 ./solenv (build/installation)
> 12 ./scp2 (installation)
> 13 ./extensions (various)
> 15 ./connectivity (Base)
> 22 ./sc (Calc)
> 39 ./sw (Writer)
>
> If we could coordinate those interested in a bit more systematic manual testing
> to look at these areas for micro releases, to prevent "fixed a regression,
> introduced a regression scenarios", that would be awesome. Since the number of
> changes in one specific area of code in rather limited, we then can:
>
> - pinpoint the change (and the developer) that causes the trouble quickly
> - reject fixes that cause another regression
> - consciously make a choice to include a fix that causes a minor regression,
>  when it fixes a major one and there is no good alternative
>
> Divide and conquer is the way to go here. With the 22 commits in Calc for
> example this should be something one volunteer should be able to comfortably
> have an eye on over the ~one month period between both releases. Writer (most
> likely because of Michael Stahls regression fixing rampage) currently might
> need two people volunteering. And obviously, the number of commits will likely
> slow down now after 3.5.1.

IMHO the way to go is automated testing and adding a test case for
fixed bugs im possible. I fear that most of the time not even devs
know all affected features so how should a normal user or qa person
know what to test.

>
> So dear QA-list members:
>  Anyone interested in picking this up for one application?
>  Who would like to do it for Writer?
>  Who for Calc?
>
>> b) Regression "Bug 45385 EDITING: Copy Paste formula to different
>> document adds source document filename to references" (what even
>> isn't accepted as a bug, but will be a showstopper for me to use
>> LibO if no solution will be found) frequently forces me to use
>> 3.4.5.
>
> So far this is not a bug, but intended behaviour. As that I wouldnt consider it
> a regression or an indication of QA problems. It indicates something else
> though: we had a failure to communicate there, the intended change of behaviour
> would have needed to be communicated clearly and before the actual change was
> made. This is something we need to improve upon, and we need to find a good way
> to make this information accessable. This could be done on the dev ML, but
> given the noise on it a bug filed for this would be even better. I am open to
> other ways to improve communication there, if proposed.

The old behavior while making sense in some corner cases was build on
a broken design. Even the OOo Calc devs knew that and added a big
warning in the code and had the code ready to show a warning message
to the user that the copy/paste action with absolute sheet refs
between different docs is not stable. I fear they just never activated
the warning message.


And some more general remarks.
I think it was always known that the .0 release will still contain
some bugs and maybe the .1 release too. I think we are doing a great
job in the 3-5 release in fixing the serious bugs so that 3.5.2 is
already more stable than 3.4.5 or 3.4.6 (at least in calc). For
example my query for calc bugs shows no serious calc bug that is open
and that can be fixed for 3.5.2.

What is annoying me is the steady complains about every changed
feature and fixed bug as making it worse. We sometimes need to change
the behavior a bit to fix bugs. Instead of immediately screaming I
would love to see some more constructive critisism especially here on
the qa-list instead of indirectly saying that the devs ruined the
release with this change. I will now stop here.

Markus