[gst-devel] state of the union

[Thomas Vander Stichele]

Hey gang,

First of all, Happy New Year.

Second, I want to apologize to those I have irritated over the past
month.  I was getting more and more frustrated at having the feeling we
had no idea where we're going and no control of what was being broken. 
Discussing things with Christian over the past week has made me realize
a few things, and he also reminded me to lighten up and not take things
so serious all the time :) (Thanks, Christian)

On the other hand, we did have some issues, and I want to try and
summarize the problems we had and offer some ideas on how we can avoid
the problematic bits in the future.  We might not agree on everything
but I think all will agree that everyone has had some things irritate
them or block them in the past months.

* MAINTAINERSHIP

We discussed this both through mail and IRC, without any clear
resolution.  Long ago, we had reasonably clear maintainership in Erik. 
We didn't always agree, but since it was his project he had final say. 
There were issues (I remember as a newbie being responsible for getting
core and plugins split up, and it did take some flaming to get there. 
Hey, I was new), but mostly things rolled along nicely.  

After Erik went away, Wim sort of naturally took over.  Since he was
writing most of the code, doing the design, and so on, people accepted
him without problem.  It also helped that beside being a good coder, Wim
was very helpful in resolving end user issues as well.   When Christian
had a file that didn't play anymore, Wim would drop what he was doing
and look at that instead.

So, Wim and Erik had the qualities people have said they look for in a
maintainer: trusted by people, fairly easy to work and get along with,
knows the code well, and does active development.

After Wim left on a break, things were less clear.  People kept on
working on things they used to work on before, but there wasn't a clear
encompassing maintainer.  Sure, someone did most of the work on the
core, someone on the website, someone on the releases, someone on these
plugins, and so on.  But almost all of the problems we were unable to
resolve easily or nicely crossed those unspoken boundaries and incited
big discussions.  All of those discussions left some people without a
sense of resolution, since there wasn't a clear maintainership.

I think that pretty much all of us feels that there is a problem we need
to solve.  Last time we discussed it, again we came to no solution since
we started from the assumption that we need one person.

So, my proposal is very simple.  Since we cannot find a person today
that has everything he/she needs to be a clear maintainer, we split up
responsibilities.  I see this as the only way we can get things decided
and moving forward in the near future.  All we need to do is decide on
some sort of list of responsibilies we want to share among eachother. 
This doesn't have to be terribly strict; it's just a way of ensuring
that when we have conflicting views, we have a way of quickly resolving
issues instead of letting them fester forever and drain everyone's
enthusiasm.

The maintainer of the responsibility would have final short-term say on
resolving matters.  Of course stuff can still be brought up, but you
would need to convince the maintainer of that part, or the others, that
your idea really is better than the other guy's.  A few things I would
consider as responsibilities to be maintained are :
a) website
b) documentation
c) core design
d) core code
e) sets of plugins
f) build maintenance
g) packaging
h) releases
i) cvs maintenance
j) cvs decisions, policy, branching
k) schedule/planning/goals
l) coding style
m) tests
n) platform maintainers 

I'm sure there are more we're missing, so feel free to propose others. 
I don't expect everything to be completely clear-cut, or to find a
maintainer for each of the items immediately, but I do hope that we can
get some people started and give them a clear sense of ownership.  I
don't want the general lack of one maintainer to keep stalling things
needing to be decided in particular areas.

In practice, it doesn't mean we all of a sudden have laws and things
have to be more strict.  It's more of a mechanism to resolve problems
when the problems arise, instead of letting them lay around.  For
example, a maintainer of something would have the natural right to
revert commits in his area if he feels they aren't complete, not done
right, need more discussion, ...

There are some people who naturally maintain plugins and get annoyed
when someone else touches them and changes everything.  People work in
different ways, and we should ensure we can work together well without
cramping everyone's style.

Also, if it's not clear, in my mind it's quite alright for more than one
task being taken on by a person.  IE, A core developer can of course
still want to maintain some plugins, and so on.

I think it's important that we try this split of maintainership since we
have no other workable alternative.  It might not be the right solution
(though I'm convinced it is), but we don't have any other right now. 
I'm sure we can make it work.

* SCHEDULE

We probably need some sort of schedule online where we plan what is
going to happen and what we are collectively targeting.  This would
include planned releases, necessary releases and targets (like "0.8
should be finished to be included with GNOME 2.6 rc1), but also fairly
impacting changes (like "run indent on the source code with our given
style right before freezing and releasing 0.8" since that's the best
time to make a huge change that breaks diffs with things before that)

I can throw something together for this in the website.

* CORE VS PLUGINS

There's been a lot of discussion due to the number of changes in the
core and so on.  I realize fully that a lot of great changes have been
made to the core.  I'm pretty excited that we can deliver interactivity,
tagging support, bidirectional caps negotiation (which buys us resizing
and audio rescaling on the fly), and so on.  I realize that there are
things in the core that still need to be done, and people doing that
work need a fair amount of freedom to do that in.  I realize that for
people like me not actively working on the core to be complaining all
the time about breakages doesn't really help core hackers to get the job
done :)

 People on both sides have said that we should probably decouple a
little more, allowing people on both sides to do their work without
getting in each other's way.  The side benefit would be that we get to
stabilize the interface between the two more.  What do you think ?
Should we make a clearer distinction between the two, decouple release
schedules finally, and shuffle our resources a bit around that split ?

* WEBSITE

I've been changing the website a little recently.  The goal is to not
use databases anymore.  I'm also trying to remove all php since we don't
really need php for what we're doing.  The goal is that if you have a
local checkout of the website from cvs, and you change stuff in it, and
it works locally, you can commit, which automatically updates the
website (this already works right now on fdo).

The website will be split in two parts; content managed from CVS, and
content that shouldn't be in cvs, like release tarballs, documentation,
media files, and so on.

Erik owns gstreamer.org and had proposed to host the website as well. 
People raised some objections, so right now I think we should do the
following:
a) deprecate gstreamer.net since it's going to take us some time to get
the domain back, if at all
b) promote gstreamer.org as the official website
c) have it hosted on freedesktop.org (that's not a problem at all at
this point)
d) get freedesktop to catch gstreamer.org, and Erik to make it point to
there (shouldn't be a problem either).

What do you think, is this fine ?

* CVS DECISIONS

Ronald and Julien have complained that the amount of breakage in the
core has stalled their development too much.  They were effectively
unable to work on gst-record and gst-player due to a broken core.  On
the other hand, core developers want to be able to make the changes they
want to make.  I think David has shown that it is possible to develop
the bulk of an invasive change on a branch.  Whether that branch is HEAD
or not is another matter, but it is possible, and from the app/plugin
developer's perspective, it is a necessity to get work done.

I did however think it probably wasn't smart, in hindsight (and I hope
David agrees) that the final commit was done before David went on a
five-day break.  We probably should have some rules of thumb for CVS
that would include something like "don't merge a branch before going on
a holiday".  While David was gone, everyone was really nervous; as soon
as he came back and started fixing things, everyone calmed down again. 
Let's avoid the periods of nervousness next time, lest we be called cats
again :)

There are more simple rules of thumb we can learn from other projects,
documentation, experience and common sense.  Which ones would make sense
for us ?
--> add your rules of thumb here.

* PLUGINS

Some people like touching every file of code there is, and some just
like working on a few and doing those very well.  Both styles make
sense, but are intrinsically conflicting.  I would propose that, for the
plugins where there is clear maintainership, that maintainership would
be acknowledged.  For example, it is clear that Ronald has done a lot of
work on v4l and ffmpeg, and Julien has done a lot of work on ximagesink
and xvimagesink.  So it's only natural they get maintainership and final
say on anything that changes there.

For that to work well, it needs to be easily visible who maintains what
and what the policies are.  Stuff like coding style for example isn't up
to the plugin maintainer, it's a project-wide issue.  But any sort of
bugfixing, reorganizing, rewriting is in the maintainer's domain.

We have said repeatedly it'd be nice to have a document for each element
describing what it does, who maintains it, what the status is, and so
on, and we probably should get that done sometime soon.  I have some
ideas on that related to the website as well.

Also, this won't work well unless the core developers send out clear
documentation on how to fix plugins, instead of trying to fix up
everything by themselves.  (this isn't a statement about how it was
done, it's just a statement on how I think we should do it in the
future).  Core developers should probably try to get their stuff right
and clear enough to explain to plugin developers, if we want to make
sure that external people will have any hope of understanding.

* RELEASES

First of all, we need to release more often.  That's pretty clear. 
Second, our releases should be less critically interdependent.  That
means I think we should decouple our core and plugins a bit more. 
Thirdly, we will have to make our source more manageable.  The plugins
really are getting too big.  Each time I bring that up, with ideas on
how to go about splitting things,  I get a backlash from other people. 
However, it is also clear that those people really don't spend much time
on making sure that their commits don't accidentally break the build.
Quite a few recent commits were obvious build breakers, indicating the
commiter either didn't realize this was affecting the build or didn't
test at all anyway.

The number one criticism of gstreamer in jhbuild is the fact that ffmpeg
breaks our build in 95% of the cases.  We can argue about htat for a
long time, but the sad fact remains that people using jhbuild don't care
why this happens, they just don't want it to happen. 

 Conversely, I don't like it either when some module I don't directly
care about other than using it manages to break a complete compile early
on.  We should adapt to what we target and not the other way around. 
This means we should split this off, and ensure it works well as a
separate entity.  As a bonus, this means it is actually maintainable
buildwise, and allows us to release this module whenever we want to
(which means, whenever someone updated original ffmpeg source so new
formats are available to gstreamer).  It is better to be able to
decouple this from the plugins release schedule.

Furthermore, a recent comment by Havoc on a GNOME list saying that
"gnome obviously cannot host mp3 decoding code at all" made me realize
that gst-plugins as currently distributed is not what can go in into
GNOME 2.6 *at all*.  Whether we like it or not, it is not acceptable to
put ffmpeg in any code form on gnome servers and get it distributed. 
You can get mad (pun intended) about that in general, but it's a real
problem that needs to be solved before 0.8

--> comments go here

* CODING STYLE

based on all of your comments, I will try and find the right indent
magic, or some other tool, that keeps us to stick to this from 0.8 on. 
There are ways of making this automatic through cvs commits, and I'll
look into that as well.  The focus will be on getting out of the way of
developers, but making sure we have consistent source.

* BUILD MAINTENANCE/TESTSUITE

I hope to expand on these a bit more.  The testsuite since a few days
manages to build all five modules succesfully on one of the machines,
yay ! It seems to me we also take more care in keeping the build going
because of it, so we'll keep it around.  I have played with the idea of
setting up tinderbox, but it seems sort of overkill, and I don't really
like the quality of tinderbox2 either.  Do people know something similar
we could use ?
The testsuite can be finished now that stuff plays again.  I hope to
compare results between the 0.6 branch and the 0.7 branch as well.

As a general rule, I would propose that any invasive change needs to
keep both the build and media test suite going.  If something changes in
those, we need to figure out first what has broken and fix it before
moving on to other rewrites.  What do you think ?

* ADD OTHER AREA HERE

I'm too tired and supposed to do some real work too today, so I'll stop
here.  Hopefully we can get some decisions made on how to go along from
here.

Feel free to flame me !

Thomas

Dave/Dina : future TV today ! - http://www.davedina.org/
<-*- thomas (dot) apestaart (dot) org -*->
I'm alive.
I can tell because of the pain.
<-*- thomas (at) apestaart (dot) org -*->
URGent, best radio on the net - 24/7 ! - http://urgent.fm/