Best git like versioning tool for many small files

Question

I was wondering what everyone's thoughts were on the best 'git like' versioning tool that can be used for many small (1 - 30kb) files. All I really want to track are deletions, moves, renames but not content changes.

My situation is about 200,000 1 to 30 kilobyte files.

Some searching brought me to git-annex. Will that do what I need?

git was made to manage source code. those files are small too... — Janus Troelsen, Jun 16 '12 at 17:36
git keeps track of deletes, moves and renames. With git log you can '--follow' single files — peshkira, Jun 16 '12 at 21:28
re: Why not git itself? Git doesn't scale well for many small files or large files @see http://stackoverflow.com/questions/984707/what-are-the-git-limits/984973#984973 — Matthew Kirkley, Mar 25 '21 at 18:35

score 1 · Answer 1 · answered Jun 17 '12 at 02:08

1

git-annex is the only one where I know - based on git. This tool supports what you are looking for and more. e.g syncing multiple annex repo's in one shot. You have full git command support and all new annex commands.To clarify, rename, delete, move appears to git. Annex use symbolic links for tracking your content.

I use the tool to track and update my homepage files. Rsync can't handle rename. That was the reason why I use this tool, and because I know git well.

answered Jun 17 '12 at 02:08

theforce

1,505
1
11
13

2

Git-annex does not provide many (if any) advantage over git itself in this context (many small files, see my comment on the other answer). Instead you are even giving up the central feature of git: version control. – Johannes Gerer Oct 29 '12 at 03:39

score 1 · Answer 2 · edited May 23 '17 at 12:24

1

To reiterate a comment, why not just git itself? The tool was designed to work well in many different contexts.

Though, for what it's worth, tracking actual renames is not so easy in git. The tools are there, but there's no easy way to access them. See Find git commits that "rewrite" files.

Another idea: find some simple tool that gathers all file names into a text file and just use git to track that. Again, might handle renames so well, though.

edited May 23 '17 at 12:24

Community

1
1

answered Jun 17 '12 at 02:26

asmeurer

86,894
26
169
240

Git doesn't perform super well in a situation where you have large files or many tiny files (@see http://stackoverflow.com/questions/984707/what-are-the-git-limits/984973#984973) – Matthew Kirkley Jun 18 '12 at 00:55
1

True, but git-annex does not solve this particular problem. Since every file handled by git-annex is embedded in the underlying git repo and thus the number of files on both solutions are identical. – Johannes Gerer Oct 29 '12 at 03:37

Best git like versioning tool for many small files

2 Answers2