[Libreoffice-qa] 3.5.0 QA ... from BHS 1 to BHS 2

[Michael Meeks]

On Mon, 2012-01-09 at 16:20 +0100, Nino Novak wrote:
> Just some gut feelings from an occasional tester.

	Good to understand these, thanks for sharing :-)

> First, somehow I'd personally like to have a longer period for Release related 
> QA activities/events, at least a one-week period after each beta or RC 
> release. (Even better 2-weeks, though this might rise distraction.)

	So - I'd love to understand this desire for less frequent releases
better :-) After all, we have tinderboxes churning out at least daily
releases (in theory), perhaps several a day if we are lucky.

	What is the concern about having new RC's ? is it that you think
developers will not care about and/or test any bugs that appear in
something one release-candidate old ? [ that seems unlikely if it is a
serious bug ], or ? ...

> Fourth, which is more an open question, how the success of Release QA could be 
> monitored intelligently. My (naive) wish would be to have usage numbers, let's 
> say
> - how often a Release has been launched on which OS platform without failure

	We have some download statistics of those that can be extracted (I
suspect), and we have the on-line update statistics too which may give
some yard-stick for successful launch ;-) usually the app has to stay
alive for a little while to do that request.

> - how often which module has been started
> - how many documents have been created/edited/viewed successfully
> - which particular functions have been called how often successfully

	These other phone-home things are more tricky, needing coding support,
but it's of course a good idea to ensure good code coverage. Ideally -
I'd like to reduce the burden on human QA though, so we're investing and
encouraging (where we can) fast automated test that run during the
compile: so you should never get a build that has pathaological failures
[ assuming our test are complete enough ;-]. Hopefully that makes the
process of QA more difficult & rewarding ;-) but of course there is
always room for lots of improvement, and some things are hard to test.

	One thing that is really nasty to test is the new
header/footer/page-break stuff. I get intermittent leakage of
page-breaks in documents (with several rendered on the screen); -but-
while (after editing a document) I can reproduce them nicely, if I save
& re-load in another instance - I cannot ;-) so - there is a real need
for some "from a clean document" reproduction steps for those issues -
some of which may be races too ;-) help there much appreciated.

	ATB,

		Michael.

-- 
michael.meeks at suse.com  <><, Pseudo Engineer, itinerant idiot