[Mesa-dev] Moving amdgpu/addrlib into a git submodule

[Mathias Fröhlich]

Hi,

On Wednesday, 10 August 2016 15:19:06 CEST Nicolai Hähnle wrote:
> On 09.08.2016 22:12, Dave Airlie wrote:
> >>
> >> tbh, git submodules are more annoying than they need to be, and I'm
> >> not really terribly excited to use that for something that is a build
> >> dependency.  Maybe just move it into libdrm instead?
> >>
> >
> > I've only had to use git submodules once with spice project, and it
> > was a nightmare. It makes packaging etc a real pita.
> 
> It would be nice to have something more concrete to go on, like what are 
> the problems _really_ :)
> 
> 
> > Alternatives are something like a fetch external sources script,
> > that does git submodules but does it better, you'll see Vulkan-CTS
> > etc use something like that, it would have to be integrated with
> > the build system a bit better though.
> 
> What pain do submodules give you that wouldn't be even _worse_ with a 
> separate script for fetching external sources?
> 
> 
> Look, there are a bunch of different use cases here. Distro packagers; 
> Mesa contributors and bleeding-edge users; and us (which is slightly 
> separate because we're not _just_ Mesa contributors but have parts of 
> the stack elsewhere, which is why this is coming up in the first place).
> 
> Most of the complaints seem to come from distro packaging, though I have 
> yet to understand what the exact problems are.
> 
> For Mesa contributors and users, I'd really like to avoid the hassle of 
> having to get another repository manually.
> 
> For us, in addition to avoiding the same hassles there's the question of 
> maintainability, and perhaps (with a huge load of optimism) getting the 
> closed source folks a bit more involved in the fact that there's another 
> world out here as well.
> 
> On those axes, I've seen one alternative suggestion that makes sense to 
> me, and that is subtree-merges. I have to play with them a bit, but the 
> initial impression is good. They seem like a more git-like solution. 
> That would also be something new for Mesa, though, since it would mean 
> more exceptions to the otherwise linear history.

For subtree-merge:
There is a pair of visualisation teams that used to work for the same company until 
they both got sold to individual vendors but still wanted to share their basic framwork 
code. Those two applications done by these teams are somewhere beyond - may be 
far beyond 1e6 loc each. The shared part is at least 2e5 loc, probably more. I do not 
recall the exact numbers. The shared part is communicated via a common upstream 
repository managed by git-subtree. There happens branching merging, rebasing, the 
usual git things. The git version that is used is ancient, the one from redhat 6.
The git repository is based on an import from svn with non standard branch names in 
svn and that svn used to be imported from cvs. All together more than 10 years of 
history.
Observed problems: It needs special care to be handled. There were not so nice things 
that happened like disconnected git object trees. After installing a git hook in the 
subtree managed upstream repository that refuses such a push, disconnected object 
trees never happened again. Also checking out commits from the subtree side of a 
subtree merge gives interresting results. Well, plausible once you think about it, but 
probably surprising for the average git user. You will see a completely different 
filesystem structure, the one of the subtree you have.
The success side: A hand full years sharing > 2e5 loc where normal development 
happens. Probably some orders of magnitude more changes than I observed with 
addrlib since its creation. It basically does what you expect when you think about the 
problem to be solved. So, beside the problems there has been loads of expected and 
good behavior. Without subtree merge that undertaken would have been much more 
difficult to infeasible.

I do not think that I can give a recomendation. The basic constraints like the age of git 
used and the history of the repositry may introduce problems that you never observe 
with something simpler.

best
Mathias
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20160811/05130b53/attachment-0001.html>