[Mesa-dev] so the development model is working?

[Brian Paul]

Dave Airlie wrote:
 >
 > So in the spirit of being less of a dickhead,
 >
 > Lets have a development model discussion. Can anyone who has an
 > interest in the development model please answer the two questions
 > below in their view/opinion.
 >
 > a) what are the goals of the mesa project and development model?
 >
 > to produce drivers for graphics hardware.
 > to produce something that Linux distributions can consume in a useful
 > manner. Like if you want to know where mesa is shipping most units,
 > its in Ubuntu and Fedora, there is probably 100-1000 more copies of
 > mesa shipping in these two distros than everything else ever produced
 > by this project.

Sounds good.


 > b) how does the current development model impact those goals.
 >
 > The current development model seems to be designed around the VMware
 > working model, not the distro working model. I'm not sure what to call
 > the VMware working model other than that. I call it that because we
 > used to have the TG working model and its not the same as that, and
 > its not the same as any other Linux shipped projects working model.
 >
 > (The TG model didn't work so well either but it was part of TG's
 > business model and we had a smaller community back then, it mainly
 > consisted of driver code drops appearing at misc times and releases
 > happening on a whenever we have time approach, this was fine then I'm
 > not sure with it now, also CVS sort of drove a lot of the problems.).
 >
 > So whats wrong with the current model. It basically seems to suggest
 > that everyone working on mesa should be working on the stable branches
 > and not mainline. Why? because if I add a new feature to my mainline
 > branch then happen to notice a bug fix along the way, then I have to
 > stop what I'm doing, check out stable, run a test suite, fix the bug
 > in stable, run another test suite, merge stable to master, run a test
 > suite on master before my fix, run a test suite on master after the
 > merge, hope I didn't break 5 other drivers doing the merge. Then I can
 > keep going working on my feature, the other key problem is I as a
 > developer have to understand what is suitable for stable and what
 > isn't. I don't think every developer needs to be responsible for this
 > decision. It also makes the merge history hella ugly but we can get
 > over that.
 >
 > It means we lose the ability to have stable driver maintainers, as
 > cherry picking isn't allowed. So for instance I have 2-3 developers
 > who are stabilising master as hard as they can for the next release,
 > I'd like to out-of-line pull some of those fixes into stable and not
 > have the world end. If every developer has to maintain development and
 > stable trees its a major duplication of resources vs one developer
 > once every couple of weeks just pulling back a big load of fixes to
 > stable, and doing a regression run across them, and bisecting out any
 > regressions before pushing the whole lot to stable. Now I can
 > cherry-pick a bunch of fixes to 7.8 now off-line, but I still end up
 > having to merge that back to master and retest master for breakages,
 > instead of concentrating on making the stable backport work properly,
 > so it makes me waste a lot of the limited resource which is developer
 > time.
 >
 > So here is the thing I can totally see how the current driver model
 > would be advantageous to the subset that is (a) drivers that don't
 > ship in the distro channels, and (b) drivers that have a lot of
 > development resources where developers can live on the stable branch
 > for long periods of time. However I question whether this is
 > optimising for the right set of people, considering the number of
 > units of mesa shipping in the distro channel and the overheads to
 > community driver developers.
 >
 > Dave.


Dave,

I'm sorry you're frustrated, but let's see if we can at least come to 
a better understanding of where we're each coming from.

Your message seems to boil down to "cherry-pick fixes from master back
to the stable branch" vs. "fix bugs in the stable branch and
periodically merge to master".

If you have issues with the timing of releases, take it up with Intel.
They're the ones who suggested the quarterly release model.  I've been
OK with that so far, but I don't think it's ideal.  Sometimes the
quantity and collection of changes/fixes don't neatly fall into the
calendar months.

I'd like to respond to some of your arguments:

"""So whats wrong with the current model. It basically seems to
  suggest that everyone working on mesa should be working on the stable
  branches and not mainline."""

Of course not.  Many of us are working on new features or redesigning
things.  That's what master (and feature branches) is for and is
totally inappropriate for stable branches.


"""because if I add a new feature to my mainline
  branch then happen to notice a bug fix along the way, then I have to
  stop what I'm doing, check out stable, run a test suite, fix the bug
  in stable, run another test suite, merge stable to master, run a test
  suite on master before my fix, run a test suite on master after the
  merge, hope I didn't break 5 other drivers doing the merge."""

Hmmm, here's how I work:  I have several separate git check-outs in
different directories.  At the moment I have four that I'm actively
working on (plus ~30 inactive/older ones).  My active directories
contain different branches (master vs. 7.8 vs. 7.7 etc) with different
builds (swrast vs. DRI vs. llvm, etc).  I typically have a terminal/tab
open on each of my active check-outs.  This lets me quickly and easily
compare the results of Mesa 7.7 vs. 7.8 vs. master when I find a bug
or want to compare performance, etc.  Do you really do all your work
in one directory with just one check-out??

I'm usually working on master.  If I think I've found a bug (or
someone reports a bug in 7.8, for example) I just change terminal tabs
and start debugging in my 7.8 directory.  I don't have to save my work
on master or check out a different branch, etc.  Just switch to a
different terminal and fire up emacs.  After I fix the bug, I commit
it to the 7.8 branch and document it in the release notes file (I'm
practically the _only_ person who bothers to do that, btw!).  If I
think it's important to get the fix into master quickly (or there's
any doubt about applicability) I'll do a merge right away.  Otherwise,
I just resume whatever I was doing before, knowing that the fix will
propogate into master whenever we do a merge.

I think you exagerate how many testing steps you have to do for each
and every fix.  Absolutely, there are times when thorough testing is
needed.  But lots of simple bug fixes/changes in 7.8 can be merged
into master without worry.  I can provide many concrete examples if
you don't believe me.


"""So for instance I have 2-3 developers
  who are stabilising master as hard as they can for the next release,
  I'd like to out-of-line pull some of those fixes into stable and not
  have the world end."""

What is stabilizing?  It's finding/fixing bugs and running tests,
right?  That's what the stable branch is all about so why not do that
work on the stable branch from the get-go?

Master is the place where new work is done: new features,
re-architecting, moving things around, etc.  Those changes (and new
feature branches) can come along any time and potentially effect all
the drivers and invalidate any testing done moments before.  Things
get broke there all the time (that's inevitable - no matter how hard
we try, nobody's a perfect coder).  Is that the place where you want
your people trying to stabilize things and fix bugs?  Really?


BTW, let's take a look at the changes to the 7.8 branch.  There's
approximately 85 commits on the 7.8 branch since it was created.
Who's doing the work?  Let's see:

                  Author             Signed-off-by
Brian Paul         20                    18
Jeremy Huddleston  18                    17
Ian Romanick        6                     1
Dan Nicholson       4                     4
Vinson Lee          4                     0
Jesse Barnes        4                     0
Matthiew Herbb      4                     0
Pierre Willenbrock  3                     0
Alex Deucher        2                     0
Aaron Plattner      2                     0
Chia-I Wu           2                     0
Maciej Cencora      2                     0
Michel Danzer       2                     0
Kristian Hogsberg   1                     0
Ben Skeggs          1                     0
Dave Arlie          0                     1
(plus a few other one-timers)


Also, there's been six merges of 7.8 to master so far:
   Brian Paul         3
   Jakob Bornecrantz  1
   Ian Romanick       1
   Michel Danzer      1


If you're concerned about producing a stable driver, why aren't you
making more fixes to the 7.8/stable branch, whether by cherry picking
or whatever?  That's the whole point of it.  Master is not a stable
branch.

Look above and see if you can guess why I prefer doing merges to
cherry-pick.?  I'd rather do 3 merges vs. 20+ cherry-picks.  Cherry
picking quickly becomes a PITA once you get beyond a handful of
patches or one commit per week or so.

Really, I get tired of hearing about the implied pain of fixing things
in the stable branch when I'm the person doing most of the work to
maintain that branch.


Dave, I realize that you (and others) feel strongly about your way of
doing things, but so do I.  Plus, I'm doing more to try to maintain a
stable branch than anyone else.

As the project founder/leader I walk a fine line between making rules
and being flexible/open to accomodate others.  For example, as much as
I wish Mesa had a uniform coding standard, I let people who develop
various drivers use their own style since it's their body of work and
I respect that.

I think I'm a pretty easy-going project leader, I seldom, if ever,
tell people what to do or not do (and never brow-beat or insult them
as some other big project leaders do).  I don't want to tell you what
to do in this case either.  You've done a lot of great work on Mesa
over the years and I respect your opinion.  At this point, for now, I
just hope you have a better understanding of where I'm coming from.

I also hope the other developers will think more about getting bug
fixes into the stable branch, one way or another, please.

-Brian