[Libreoffice-qa] QA Meeting Minutes - 2014-12-03

[Robinson Tryon]

On Mon, Dec 8, 2014 at 3:43 PM, Tommy <barta at quipo.it> wrote:
> if we don't mark it as FIXED WFM how do we distinguish a NEW bug without a
> fix from a NEW bug which is FIXED in master but not yet present in the
> stable branch?

Sure -- and this highlights a granularity problem we have with current
bug reports: We only have a binary switch to indicate if a bug has
been fixed or not.

We currently use the Version field to point to the oldest LO version
on which we can reproduce a bug. If we had a 2nd Version field, we
could use that to point to the newest LO version on which we can
reproduce, but IMHO that's still not as granular as I'd like. I've
some cursory ideas for how we can more clearly indicate which
environments (LO version + OS + etc..) reproduce a given bug, which we
can revisit after the Bugzilla Migration.

> I think the scenario Robinson depicted is logical.
>
> 1- QA finds that a NEW bug in stable branch is somehow fixed in master so it
> marks it as WFM
>
> 2- a bibisect attempt is done to identify the committ that fixed it
>
> 3- once the bibisect is done, a backportRequest is added to whiteboard and
> devs are contacted

While we do have other work in the QA task pool (triaging incoming
bugs, bibisecting open regressions, etc..), let's see how many fixed
bugs get tagged with bibisectRequest, and see how quickly we're able
to address them. If the number is small, it shouldn't represent much
of an additional burden for QA.

If we just tag a bug with bibisectRequest and resolve it as WFM, I
think bibisecters might ignore it, so I suggest that we tag it
bibisectRequest + backportRequest at the same time. To help us keep
track of these bugs, I've added two entries to the Useful Queries
page:
https://wiki.documentfoundation.org/QA/Bugzilla/Useful_Queries#Common_Lists_of_Bugs
1) Backport requests that need bibisection
2) Backport requests that do not need bibisection (i.e. needs dev attention)

How do we decide which bugs to tag? I suggest that for now we only tag
backportRequests if the bug was reported against an active Still or
Fresh branch.

So to put that into context with our current builds, here's my proposal:

If you test a bug reported against 4.3 and find
1) It's fixed in master, AND
2) It's not fixed in 4.3.5.1 (the latest build on the 4.3 branch)

Then change status -> WFM, and add "bibisectRequest backportRequest"
to the Whiteboard.

For all other cases, if it's fixed in master, then (just) change status -> WFM.


Best,
--R

-- 
Robinson Tryon
QA Engineer - The Document Foundation
LibreOffice Community Outreach Herald
qubit at libreoffice.org