RFC: group maintainership for misc drm trees

[Dave Airlie]

On 15 February 2016 at 20:06, Daniel Vetter <daniel.vetter at ffwll.ch> wrote:
> Hi all,
>
> I've already chatted with some of you in private, here's the entire idea with a
> bit more thought. My motiviation for group maintainership of drm-misc was that I
> got a bit a guilty feeling the last few vacations/conferences when folks pinged
> about reviewed/tested pretty patches not landing. But also just increasing the
> bus factor and sharing the load better is good. And finally a shared misc drm
> tree would allow fringe drivers to get faster into Dave's drm-next by
> piggy-packing on top of the one pull request train. And it would also reduce a
> bit tree proliferation (at one point we had drm-misc/-bridge/-panel/-trivial and
> may more even). And at least everyone I chatted with seems to like the idea in
> principle.
>
> But what's still open is how to do it exactly. One big change with group
> maintainership is that you can't rebase a tree anymore. And right now I need to
> rebase drm-misc fairly often to throw out bad apples again. I think solving that
> is the important bit to make a shared drm-misc work. A few ideas:

This is kinda what I don't like. I don't want a tree that can't rebase
out bad stuff
coming to me that often. If we are being too over eager in merging
stuff the answer
is to merge less not try and merge more. If we have a tree where things
are thrown until they stick I'm not sure the history will ever be nice
to pull from.

I'd rather consider a staging tree where everything from patchwork can
get thrown
into, CIed, but that we cherry-pick fixes things to go back to me from the what
works category.

I'm finding with i915 for example there is a massive latency in the pipeline now
waiting for fixes, and the pipeline to the end of drm-intel-next is
very long and
hard to figure out what fixes should be pulled back, and how much of the
driver has been rewritten between the -next and the -fixes pulls.

> - I think CI is super-important. We're starting to finally roll that out for
>   real for i915, and it's catching an awful lot of stuff already. Not yet ready
>   for prime-time on public mailing lists, and for misc we probably can't test
>   every patch before they land like we do for i915. But CI should have veto
>   power before a pull request goes to Dave imo.
>
>   For non-i915 Daniel Stone and others are working on ARM CI using the generic
>   igt testcases for kms. And I'm open to merging driver-specific tests into igt
>   too, if it makes sense, and e.g. Eric has already pushed some vc4 tests.
>
> - Stuff needs to at least compile cleanly before pushing. I've been really bad
>   at that wrt arm drivers with my own drm-misc, but turns out it's fairly easy
>   to get this right: http://blog.ffwll.ch/2016/02/arm-kernel-cross-compiling.html
>
> - Bad apples need to be kicked out with reverts, not rebases. I think that's
>   fine for simple patches, and hence those can go directly to drm-misc. But a
>   bunch of subsystem-wide refactorings go in through mis trees, and for those
>   constantly mass-reverting until it's all solid is silly. And ime you need some
>   soak time in a shared tree to iron out bugs with those kind of endeavours. We
>   can address that with ad-hoc&short-lived topic branches which are then again
>   owned by a single maintainer, but automatically pulled into an integration
>   tree. After some soaking time to give CI systems time to crunch through those
>   topic branches can then be merged into the main drm-misc and removed.

I think integration trees are probably the way forward, but I
understand they come
with a massive overhead for constant merging, I like the idea of topic
branches until
it becomes my turn to spaghetti merge 3-4 features at once. At that
point I still
fallback to the slow things down, pick one feature merge it, back the
others off,
and I think this is something we should do more off, the pile everything in and
hope that magical CI will stabilise things doesn't seem to be a responsible
process to me.

so I'd like CI to be happening but I'd like it to be happening before the git
history is baked into stone.

> - Also we can't roll forward to latest drm-next easily with rebases any more. So
>   that needs to happen either right after the pull request lands (when no patch
>   has been merged meanwhile). Or with backmerges, which then need a short commit
>   message as to way the backmerge is needed ("Backmerge because we need
>   $feature" is what I usually type), plus sob line from the committer. And of
>   course the backmerged treed must be stable/non-rebasing and preferrably the
>   backmerge should be a release/pull-request tag.
>
> - For tooling I suggest we just go with drm-intel maintainer-tools for a start.
>   Picking dim has a few reasons:
>   * Well tested, documented and fairly complete (includes e.g. bash-completion).
>   * Integrated support for topic branches, including support for git worktree.
>   * Well-exercised and robust integration tree support.
>   Short-term we could add some convenience functions (e.g. for creating/merging
>   drm-misc topic branches). Long-term we might or might not want to have this
>   separated from drm-intel - that should be possible but a bit of work. Also,
>   this way drm-misc would stay at it's current location while we figure things
>   out. Longer term we might also want to look into adding other big drivers
>   into an over drm integration.
>
> Of course group maintainership needs an initial group. My experience from
> drm-intel is that a bigger group of maintainer has benefits: It's clear that
> part-time maintaining is ok too, with maintainers focusing on their area of
> interest/expertise and only helping out in other places when there's a gap (due
> to e.g. vacations). Anyway, here's my thoughts for the starting group:
> Archit, Jani, Thierry & me as existing maintainers of drm-misc/bridge/panel,
> Alex&Rob as maintainers of big drivers and engaged in core drm stuff,
> Daniel Stone so that he has no more excuses to stall on arm drm ci.
>
> I think if this goes well we can extend it to more driver maintainers, e.g.
> Patrick would like to just push gma500 patches to some tree and not fiddle with
> pull requests all the time himself.

Is pull request sending a big issue? I don't really want a misc tree
as a way to avoid sending distinct pull requests for drivers. If we do
this I want
it to be drm core patches and subsequent fallout only and if we want
an unmaintained
drivers patches tree to do that separately.

I don't really like the misc tree replacing panel/bridge tree either,
those were nicely
compartmentalised in my head as trees I pulled but didn't really care
about as Thierry
did a good job and I don't have ARM hw that I personally care about.

It's important to me that stuff still comes in some sort of
compartments, as it decides
for each pull request how much time/effort I put into testing it on my
end, I'm pretty much
at the mercy of ARM hw maintainers, but with x86 I still like to smoke
test it on some
hardware as much as possible, also with DRM core stuff.

So I'm still not sure where I stand on this, personally I don't care
if you take holidays
and stuff stalls, delays aren't the big enemy here, merging crap is
much worse than
delaying.

Dave.