[Libreoffice-qa] Some ideas about involving people in bug triaging

[Mirosław Zalewski]

Hi

I am long-time LibreOffice user who tries to help others on our users mailing 
list. This caught an eye of Michael Meeks some time ago. He asked me whether I 
have some ideas about encouraging people to get involved in bug triaging. 
After exchanging couple of messages, I wrote one rather lengthy. 
Unfortunately, I have never heard back.

So, below is little edited message I have sent to Michael back then. Hopefully 
my ideas will span some discussion and at least some of these - modified or not 
- will be picked up.

Before we begin, a little disclaimer: I am not programmer and I know really 
little about LibreOffice QA. Given your workflow and resources, some of my ideas 
might be invalid or even stupid. Sorry in advance if this is a case; I write 
this message with good intentions.

There are two main questions that my ideas try to address:
1. How to get more people aware of bug triaging?
2. How to create steady team of bug triagers?

= How to get more people aware of bug triaging? =

This one is all about marketing and I think answering it is rather easy. 

The main point here is, that most users are not looking for getting involved 
in LO development. Yet we must reach them and somehow encourage them to do it.

I think that this strategy might work:
We must tell people that they can make a change - that they can make software 
better. This should catch their eye, because everyone wants to use better 
software. When we get their attention, we must tell them that no special skill 
is needed and everyone can help. No geeky talk about bugzillas, bibisecting, 
C++, API, building from source etc. (as I have noticed, this is already 
discussed in recent thread). We must ensure people, that task is easy 
and requires only few minutes of spare time. This should encourage people to 
try.

Now, when we got attention, there are three areas that we must cover:
a. Where do we send people?
b. How do we make them stay with us?
c. How do we reach them with our message? (OK, this one should be answered in 
the first place, but let's suppress it for a while)

== Where do we send people? ==

At this point, we got some person involved. He wants to help. He goes to some 
page, let's say <http://wiki.documentfoundation.org/BugTriage>
He click first "query" link he can find and... 200 bugs shows up. Just reading 
titles would take 10 minutes or something. Reading all bugs and trying to 
reproduce them would take hours. And we promised it is easy and not time-
consuming...

List of 200 bugs is discouraging, because people like to complete things. If 
we ask someone to help us in confirming bugs and then give him list of 200 
issues, then he thinks that we ask him to go through 200 issues and confirm 
each and every one. This is tedious task and most people will give up before 
even starting. They will feel overwhelmed.

We must protect people from any negative feelings. We should give them only 
few task to check. They will happily try.

I can think of solution. Make special page, where all bugs that need attention 
are gathered, but displayed at random chunks of five. If you see list of five 
tasks, then it looks like something easy. By each item at list there should be 
checkbox that can be clicked to mark item as done (this is pure UI, no need to 
connect with database or something). This item could go down the list, 
disappear or anything. People LOVE to see items at their to-do list erased. 
When they clear the list, they will feel happy and satisfied. And some (most?) 
of them will ask for new list, because they feel in control and they have 
proven themselves that they can do it.

Such page will work great for NEW bugs and FIXED bugs (verifying if they 
really has been fixed). But it won't really help de-duplicating. We can't take 
five random posts with some word in subject and hope they are duplicates, so 
user can easily mark them as so. Honestly, I have no idea how to find 
duplicated bugs.
Maybe someone with experience in that matter could just describe what he does? 
Then it would only be matter of writing app that eases this task.

== How do we make people stay with us? ==

Anyway, users will get bored sooner or later. You can't perform some 
repetitive, dull task for long without powerful enough motivation. In some 
situations this motivation is money, but we can't really offer it. Instead, we 
can offer people other rewards.

I can think of four kinds of rewards. Each of it would require some way to 
measure activity of each bug triager. For any of this to work, we need 
reliable way to say how many confirmed/de-duplicated/verified bugs each bug 
triager has. I have no idea if current bugzilla can be used to gather this 
data and whether this is easy task or not.
Anyway, rewards:

a) LO/TDF t-shirt for users who confirm etc. some number of bugs, e.g. 500. If 
number is set too high, people will find task impossible. If it is too low, TDF 
will not be able to produce so many t-shirts ;) .

b) Boxed version of LO with signatures of TDF's Board of Directors or someone. 
Such item would have relatively low production-cost and very high collector's 
value (because there would be few dozens of them in entire world). And people 
like to collect rare things, even if they have no market value.

c) List of month top bug triagers placed somewhere visible. This will 
stimulate competition and people will feel rewarded (they will got their name 
public; this is some kind of fame, after all, and most people dream of being 
famous). It is important to reset counter every now and then, because people 
like to compete with equal and slightly better, not much better than 
themselves. If they see that getting into top ten would require to confirm 2000 
bugs, they won't even try.

d) Dedicate release to one most active bug triager. Again, this is reward in 
fame and rare goods (how many of us got anything public dedicated to them? And 
how many would like to?) with ultra-low production cost.

== How do we reach people with our message? ==

OK, so let's go back to beginning and think for a while: where can we post 
appeal to users?

I can think of only one the best place - announcements of new versions. New 
version is something that people talk about. News sites will repost this. I am 
more than sure that LO website gets more traffic in two-three days after 
announcements than usual. Announcements already contain "success stories" of 
LO, TDF and OpenDocument (in public administration, NGOs etc.). So they can as 
well mention that reader may already drop in and help. It must be clear where 
to start. Perhaps link to another site would work best.

But there is important thing to note. If you go to shop, you pass exactly the 
same building and streets each time. You don't think about them, because they 
are too familiar. If we put call for help in exact the same place of each and 
every announcement, people will start to ignore it. News sites will post 
something like "as usual, TDF want you to triage bugs". 
So, if we want this to work in long term, then we must make people forget 
about bug triaging and remind them now and then, like every three-four months. 
It is best to put it at uneven intervals. LO/TDF must make call for help 
something unexpected, because unexpected news gets our attention and requires 
us to act upon them.

= How to create steady team of bug triagers? =

This one is much harder to answer for me, since I don't know what we really 
need from people in core QA team.
Confirming, de-duplicating and verifying bugs is covered by large number of 
non-experts (as explained above). So what do we have left? Bibisecting, 
marking as EasyHacks, assigning bugs to developers? I believe that small team 
would be enough for these tasks. Perhaps current members of QA team could 
focus only on that?

So, this is a place that needs some discussion. If I knew what do we expect 
from "steady team of bug triagers", then I might come up with some ideas.

What I do know, is that common marketing practices wouldn't work here. 
Mentioned tasks requires some skills, which only minority of people have. Our 
strategy here shouldn't be about getting as much people involved as possible, 
blindly hoping that some of them must have necessary skills. What makes things 
worse, we can't really offer people anything but satisfaction from helping. So 
we should focus on providing as smooth learning curve as possible. For anyone 
willing to be part of QA team, there should be clear message: 
- WHAT do I do? 
- HOW can I do that?  
- WHAT must I read before? 
- WHERE do I start? 
- WHO can I ask in doubt? 
- WHERE can I learn more?
Perhaps providing good documentation is a key. Sorry to be a bit harsh here, 
but some of current QA wiki pages look like bunch of unorganized notes 
(especially <http://wiki.documentfoundation.org/BugReport_Details>). They 
could really use some cleanup.


As I mentioned in beginning, I post this message with hope that it will span 
some discussion. It's not "do what I tell you"; rather they are some ideas 
that I have come up with, which I believe are good. If I can contribute to 
LibreOffice this way, I am happy to do it.
-- 
Best regards
Mirosław Zalewski