15

I have several projects that depend on the same library, for which I'd like to maintain a separate git repository to be managed with git-subtree within each project. So for example, within each project I can do:

project1$  git subtree add --prefix=lib1 /path/to/lib1.git master
project2$  git subtree add --prefix=lib1 /path/to/lib1.git master

Now in the course of working on project1, I make some changes to lib1, say lib1/file1.c, and push this back to the central repo:

project1$  git add lib1/file1.c
project1$  git commit -m "updates to lib1"
project1$  git subtree push --prefix=lib1 /path/to/lib1.git master

So far, so good. But now I'd like to update project2's copy of lib1. So I try:

project2$  git subtree pull --prefix=lib1 /path/to/lib1.git master
Auto-merging lib1/file1.c
CONFLICT (content): Merge conflict in lib1/file1.c
Automatic merge failed; fix conflicts and then commit the result.

What's going on? I know for certain that no changes were made to any of the lib1 files under project2, so why should there be a conflict here?

The conflicts are half-empty, like those reported in this question. Everything is being pulled/pushed within a single system (OS X), so I know there's no issue with line endings as suggested there.

Surely this is a common use case for git-subtree, and has a simple answer I just can't see. Please help!

EDIT: I found an unsatisfying workaround: immediately after pushing changes to the subtree, I need to re-run subtree pull:

project1$  git subtree push --prefix=lib1 /path/to/lib1.git master
project1$  git subtree pull --prefix=lib1 /path/to/lib1.git master

Even though there were no changes, it will find something, and do a merge commit. Then, after making some changes elsewhere, the conflict won't happen the second time I pull from the central repo. But if I forget to run pull immediately after pushing, the next pull will get this conflict.

So now my question is, why does this work? Is there a bug in the way git-subtree tracks pushes, or am I missing something?

Community
  • 1
  • 1
user2509951
  • 195
  • 1
  • 6
  • I'd have a look at a hexdump of the affected lines nonetheless. A CR might have sneaked in somehow, e.g. by pasting text from some other source. – chirlu Jun 21 '13 at 19:04
  • I checked the hexdump for the file at each step, there are no CR's (0d) anywhere, only LF (0a). So this isn't the problem, though it was good to check. – user2509951 Jun 21 '13 at 23:35
  • It could, of course, be some kind of bug. Git subtree is relatively new, after all. If you can come up with a reproducible test case, you could write to the git mailing list. – chirlu Jun 22 '13 at 06:41
  • Did anyone ever figure out a better workaround to this or did a newer version of git ever fix this? – dreid Feb 03 '14 at 17:33

4 Answers4

8

I actually found the proper way to do this through some trial and error.

After this command:

project1$  git subtree push --prefix=lib1 /path/to/lib1.git master

execute a fetch command:

project2$  git fetch /path/to/lib1.git master

and then do your pull:

project2$  git subtree pull --prefix=lib1 /path/to/lib1.git master
dreid
  • 397
  • 2
  • 9
  • 2
    This has to be the accepted answer. Exactly what I needed. – hidefromkgb Nov 27 '17 at 17:43
  • Thank you! This works and got me in the right track. – Dato Sep 06 '22 at 21:39
  • Depending on your workflow, or what you need exactly, you can do away with the push/fetch/pull, and use regular merge+subtree split: `git merge $(git subtree -P lib1 split)`. – Dato Sep 06 '22 at 21:40
5

There is unfortunately no good way to split commits out of the tree without giving the newly-split commits totally different commit ids. This is because they are, after all, different commits: they don't contain the parts that weren't in the subtree. That means when you pull them back in, git will see them as entirely new commits and generate a conflict.

There are two things you can do. One of the other answers suggested doing a git subtree pull right after you push. This will work, but you end up with two copies of every commit because there really are technically two sets of changes: the upstream ones (auto-generated by git-subtree push/split) and the ones in your combined project, and you are merging them together. A shortcut for this method is the --rejoin option to split/push, which adds the extra merge commit right away.

The second option is to use --squash. This also creates new merge commits, but since you merge all the changes from the upstream repository as a single commit instead of one per original commit, it causes less clutter in your history. Most people seem to like it better with --squash.

Philip Kirkbride
  • 21,381
  • 38
  • 125
  • 225
apenwarr
  • 10,838
  • 6
  • 47
  • 58
3

Well, it seems to be a bug in git-subtree. I evaluated it for my needs and gave up. Basicaly, git-subtree push alters commit messages and thus, SHA1 of commit changes. You have to pull to merge additional commits which introduce exactly the same changes but just have different SHA1 hashes due to altered commit messages. GIT handles double merges (merging same changes again) correctly, so it silently notes the merge.

Someone has to fix it!

Esoteric Screen Name
  • 6,082
  • 4
  • 29
  • 38
Borg
  • 54
  • 1
  • 3
    That explains why the push-then-pull works like it does, thanks. It's frustrating that subtree can't handle such a simple scenario automatically. Is there a different way to do the push that would avoid changing the SHA1, or are we back to using submodule? – user2509951 Jun 26 '13 at 19:51
  • It looks like it haven't fixed yet. I've got the same issue now. – zhekaus Apr 08 '16 at 17:34
1

Inside your main project, try to pull and push with squash commands always :

step 1 : subtree pull with a squash

 git subtree pull --prefix=mainProjectFolder/subtreeFolder http://bitbucket.org/repo.git master --squash

step 2 : subtree push with a squash

 git subtree push --prefix=mainProjectFolder/subtreeFolder http://bitbucket.org/repo.git master --squash

squash flag will avoid creation of new SHA1 id's for the same commit in different repositories.

Explanation : To over come this issue you can make it a convention to use the squash flag while pushing and pulling your subtree. The issue described by @Borg about SHA1 commit id's is correct, but subtree were not really built for this. You should avoid to keep the commit Id' of the subtree(library) repository in both the parent project and subtree project. And if at all you push changes from the parent repository to the subtree repository follow this statement from the documentation(line 58): That is, if you make a change that affects both the library and the main application, commit it in two pieces.

Also this video explains it when not to use subtrees. Drag straight to 11:00 minutes to find that subtree's are not the correct solution for you if :

you have constant updates to the repository,

or, if you have many dependencies,

or, if everyone in the team has to learn subtrees.

Community
  • 1
  • 1
ishanbakshi
  • 1,915
  • 3
  • 19
  • 37