[Libreoffice] [gitdm-lo-config PATCH] speed up an improve merging of logs

[Miklos Vajna]

Hi Cedric,

I'm attaching a patch that speeds up running merge-log for me. See the
commit message for details.

Here are the times on my machine:

~/git/libreoffice/clone$ time ~/git/gitdm-lo-config/merge-log > log
real    1m15.034s
user    0m46.174s
sys     0m12.736s
After applying the patch:
~/git/libreoffice/clone$ time ~/git/gitdm-lo-config/merge-log2 > log2
real    0m25.356s
user    0m23.389s
sys     0m1.893s

Also, now it outputs each commit only once, so the size is reduced:

215M Oct 25 13:45 log
After:
110M Oct 25 14:35 log

To check that the output is correct, I did:

$ grep ^commit log |sort -u > log.commit
$ grep ^commit log2 |sort -u > log2.commit
$ diff -u log.commit log2.commit |wc -l
0

So I think I hope I did not broke it. :)

One minor not is that I did not find an easy way to exclude all commits
having a corrupted commit object (in that case the date will be epoch 0
at user-level), so I just excluded those commits explicitly. Given that
you may need to extend that list in the future, I documented that
process in the script itself, so hopefully it'll be easy to understand,
even if it's git plumbing magic. ;)
-------------- next part --------------
From d8177196340b03d59af0cb7045a6eb91eea3f137 Mon Sep 17 00:00:00 2001
From: Miklos Vajna <vmiklos at frugalware.org>
Date: Mon, 25 Oct 2010 14:37:05 +0200
Subject: [PATCH] speed up an improve merging of logs

Optimize merging of the logs by creating a single repo via hardlinks.

This also fixes the output in case the merge of a commit has an earlier
timestamp than the merged commits (for example due to different
timezones). In such situations the merged commit was listed earlier than
the actual merge which is impossible.
---
 merge-log |   63 +++++++++++++++++++++++++++++++-----------------------------
 1 files changed, 33 insertions(+), 30 deletions(-)

diff --git a/merge-log b/merge-log
index 72e838b..7b41a41 100755
--- a/merge-log
+++ b/merge-log
@@ -1,36 +1,39 @@
 #!/bin/sh
 
-if test -e all-final.log; then
-    rm all-final.log
-fi
-touch all-final.log
-
-cd ..
-git rev-list --timestamp LIBREOFFICE_CREATE.. | sed -e "s:\$: ..:" >>$OLDPWD/all.log
-cd $OLDPWD
-
-for repo in `ls`; do
-    if test -d $repo; then
-        cd $repo
-        git rev-list --timestamp LIBREOFFICE_CREATE.. | sed -e "s:\$: $repo:" >>../all.log
-        cd ..
-    fi
+rm -rf all.git
+repos=".. `ls`"
+mkdir all.git
+cd all.git
+git init -q --bare
+exclude=""
+for i in $repos
+do
+    [ -d ../$i ] || continue
+    repo=${i/../build}
+    cp -al ../$i/.git/objects .
+    git update-ref refs/remotes/$repo/HEAD $(cd ../$i; git rev-parse HEAD)
+    git update-ref refs/remotes/$repo/LIBREOFFICE_CREATE $(cd ../$i; git rev-parse LIBREOFFICE_CREATE)
+    exclude="$exclude ^$repo/LIBREOFFICE_CREATE"
 done
 
-sort +0 -1 all.log > all.log.new
-mv all.log.new all.log
+# If we want to exclude a commit, we need to list its child and parent(s) in the
+# grafts file like this:
+#
+# $ git rev-list --all --children |grep ^91c969a154b4c1e5a986c809e78a26c4849b7b18
+# 91c969a154b4c1e5a986c809e78a26c4849b7b18 60f6aa831e389c46cafd71ec9f5a6d01ed89b6a6
+# $ git rev-list --all --parents |grep ^91c969a154b4c1e5a986c809e78a26c4849b7b18
+# 91c969a154b4c1e5a986c809e78a26c4849b7b18 0ccfc62e9c03f89395f4cae4a8c31bcc7df9f9ab
+# echo "60f6aa831e389c46cafd71ec9f5a6d01ed89b6a6 0ccfc62e9c03f89395f4cae4a8c31bcc7df9f9ab" > .git/info/grafts
+#
+# Note that merges have multiple parents, that's expected.
 
-while read entry ; do
-    timestamp=`echo $entry | cut -d ' ' -f 1`
-    commitid=`echo $entry | cut -d ' ' -f 2`
-    repo=`echo $entry | cut -d ' ' -f 3`
-    if test "$timestamp" != "0" && test "$commitid" != "f07fbb7f983625881c1adb97768f1d9eaf250b22"; then
-        cd $repo
-        git show $@ $commitid >>$OLDPWD/all-final.log
-        cd $OLDPWD
-    fi
-done < all.log
+echo "b85c2871a3e7637b5702ac5e92e0c77036bec05e 1f33e9500443f6c947294c0776236c4808ddd262 7a86940efec79bdf7e8c696f250c1a67378d3667
+f0c8aff64ba26179dd33db0b22c729f506ab56f2 e2060d42d0267729888d68448e668a7f9df83c3b 758f7c6604281f000c7e6756ea8e5741992a61cc
+abae0dfceff9815d8058390e28f1e84e875d6cea 10146e0e1f2a6309f2206ea6b5b9878689d60a2d
+39b15650818f7dff8a717afff5714232bae12f0c 465611c8ff92b6f6e47f683f2022b35126becf16" > info/grafts
 
-cat all-final.log
-rm all.log
-rm all-final.log
+git log --all -p $@ $exclude
+cd $OLDPWD
+rm -rf all.git
+
+# vi:set shiftwidth=4 expandtab:
-- 
1.7.3.1

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/libreoffice/attachments/20101025/f780dcc0/attachment.pgp>