[Intel-gfx] [PATCH 2/2] dim: Try to gc the rr-cache
Daniel Vetter
daniel.vetter at ffwll.ch
Wed Jul 12 12:12:24 UTC 2017
The problem is that we have a distributed cache - every committer has
a copy. Which means even just a slight clock skew will make sure that
a naive gc algorithm results in lots of thrashing around.
To fix this add a huge hysteresis: Only add files newer than 1 day,
and only remove them when older than 60 days. As long as people have
reasonable accurate clocks on their machines this should work.
A different problem is that we can't use filesystem timestamps (and
hence can't use git rerere gc): When someone comes back from vacations
and updates git rerere, all the files will have current timestamps,
even when they've been pushed out weeks ago. To fix that, use the git
log to judge old files to remove. Also, remove old files before adding
new ones, to avoid confusion.
Also, we need to teach the cp -r to preserve timestamps, otherwise
this won't work.
v2: Use git log to remove old files.
v3: Remove the debug uncommenting (Sean).
v4: Split out code movement and explain better what's going on (Jani).
Signed-off-by: Daniel Vetter <daniel.vetter at intel.com>
---
dim | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/dim b/dim
index b788edd29653..79d616cbf354 100755
--- a/dim
+++ b/dim
@@ -513,9 +513,15 @@ function commit_rerere_cache
git pull >& /dev/null
rm $(rr_cache_dir)/rr-cache -Rf &> /dev/null || true
- cp $(rr_cache_dir)/* rr-cache -r
+ cp $(rr_cache_dir)/* rr-cache -r --preserve=timestamps
git add ./*.patch >& /dev/null || true
- git add rr-cache/* > /dev/null
+ for file in $(git ls-files); do
+ if ! git log --since="60 days ago" --name-only -- $file | grep $file &> /dev/null; then
+ git rm $file &> /dev/null
+ echo deleting $file
+ fi
+ done
+ find rr-cache/ -ctime -1 -type f -print0 | xargs -0 git add > /dev/null
git rm rr-cache/rr-cache &> /dev/null || true
if git commit -m "$time: $integration_branch rerere cache update" >& /dev/null; then
echo -n "New commit. "
--
2.13.2
More information about the Intel-gfx
mailing list