How to get the folder size for the whole git history

Question

I have some folder in git with different files (little images for example). Every change/insertion of images are stored in git history forever. If that folder grows too much, for example it will be over 50mb, it can make some problems to all users of that git repo.

For some reasons I cant use git-lfs because it will make huge problems.

So the question is: how to count size of all files in selected directory in all commits in the history? Or some ideas how to achieve it?

https://stackoverflow.com/questions/8185276/find-size-of-git-repo — Saurabh P Bhandari, Dec 05 '19 at 11:49
@SaurabhPBhandari for sure I have seen that post, but it only helps to find size of ALL repo, but I need to find size of exact folder inside repo — Andrey Danilov, Dec 05 '19 at 11:56
@SaurabhPBhandari it works, but unfurtunately I did my own in parallel. Both works, but show different results. I will do a few tests a bit later and will know exactly which is working as supposed. — Andrey Danilov, Dec 06 '19 at 16:03
fyi @SaurabhPBhandari my code gave me 16 mb result, urs ~1 mb — Andrey Danilov, Dec 06 '19 at 16:04
This [gist](https://gist.github.com/magnetikonline/dd5837d597722c9c2d5dfa16d8efe5b9) could be used as reference for checking if the size is valid (oneliner [command](https://gist.github.com/magnetikonline/dd5837d597722c9c2d5dfa16d8efe5b9#gistcomment-2782586)) — Saurabh P Bhandari, Dec 06 '19 at 23:57
@SaurabhPBhandari well since my script does not work properly, guess urs is better option) — Andrey Danilov, Dec 08 '19 at 09:10

Saurabh P Bhandari · Accepted Answer · 2019-12-16T01:19:35.993

As @Mehrdad mentions in the answer for Find size of git repo, the following command can be used to find the size of the git repo :

git rev-list --objects --all | git cat-file --batch-check="%(objectsize) %(rest)" | cut -d" " -f1 | paste -s -d + - | bc

rev-list takes path as an argument. So, to find the size of a directory (ex : foo/bar) in the git repo (across all commits), you can run :

git rev-list --objects --all -- foo/bar | git cat-file --batch-check="%(objectsize) %(rest)" | cut -d" " -f1 | paste -s -d + - | bc

This will however return size in bytes. You could pipe the output to numfmt util for more human readable format.

git rev-list --objects --all -- <relative-path-to-directory> | git cat-file --batch-check="%(objectsize) %(rest)" | cut -d" " -f1 | paste -s -d + - | bc | numfmt --to=si

Note: --all includes all revisions, you can specify a branch or tag name instead.

Andrey Danilov · Answer 2 · 2019-12-06T16:07:34.987

I got result using git log and git show

RESULT=0
OUTP=$(git log --pretty=tformat:%H -- path/to/data | while read hash; do
    git show --stat --name-only $hash | grep "^path/to/data" | while read filename; do
        if [ ! -z "$filename" ]; then
            git show "$hash:$filename" | grep "^size " | while read filesizeString; do
                filesize=${filesizeString#"size "}
                echo $filesize
            done
        fi
    done
done)

echo "FINISHED"

while IFS= read -r line; do
    RESULT=$((RESULT += line))
done <<< "$OUTP"
RESULT_MB=$((RESULT/1024/1024))
echo "Directory takes $RESULT_MB mb in git repository"

But I dont know why that way shows different size than @Saurabh P Bhandari's

sometimes `git show` gives bad result, that script is not best option for all cases :/ — Andrey Danilov, Dec 08 '19 at 09:11

How to get the folder size for the whole git history

2 Answers2