1

I am trying to combine several Git repositories into one. I have the folowing individual repositories:

-mainrepo
-repo1
  |-src
  |-pom.xml
-repo2
  |-src
  |-pom.xml
-repo3
  |-src
  |-pom.xml

I want to combine them into the folowing structure in mainrepo:

-mainrepo
  |-repo1
     |-src
     |-pom.xml
  |-repo2
     |-src
     |-pom.xml
  |-repo3
     |-src
     |-pom.xml

I have tried git subtree to copy the content of subdirectories:

git subtree add --prefix=repo1 https://bitbucket.org/repos/repo1.git master

and then tried to rewrite the history with git filter-repo:

git filter-repo --force --path src/ --path pom.xml --path-rename :repo1/

This approach works, but only for the first repository. When I try to do the same steps to add the second reposity, the history for the firts one vanishes:

git subtree add --prefix=repo2 https://bitbucket.org/repos/repo2.git master
git filter-repo --force --path src/ --path pom.xml --path-rename :repo2/

I have tried the approach from How to merge N git repositories into one preserving their histories? but it doesn't preferve the history of individual files. Have also tried Merge two Git repositories without breaking file history with no luck. The history is not available for individual files as well.

Can someone please come up with a better way to do it?

Edit

I have found the solution. In fact, it is pretty simple. First I need to clone every repo (repo1, repo2 ...) Then I need to rewrite the history in every repo and move the code to the corresponding subfolder with git filter-repo. Next I need init mainrepo and merge all repos. So, I need to:

mkdir mainrepo
cd mainrepo
git init
cd ..

Next, for every repo do the following:

git clone https://bitbucket.org/repos/repo1.git
cd repo1
git filter-repo --path src/ --path pom.xml --to-subdirectory-filter repo1
cd ../mainrepo
git pull --allow-unrelated-histories ../repo1/
cd ..
Solaris
  • 11
  • 2
  • The 2nd `git filter-repo` must take into account that directory `repo1` already exists; currently it simply removes it, that why you 1st history vanished. – phd Jul 27 '23 at 12:34
  • I thought about it. But how to achieve this result? I didn't find a correct option in git-filter-repo documentation. – Solaris Jul 27 '23 at 12:38
  • I must say I don't understand what you do with `filter-repo`. `git subtree add --prefix=repo1` pull everything under `repo1`, `git subtree add --prefix=repo2` under `repo2`. Why do you need `git filter-repo` afterwards? – phd Jul 27 '23 at 12:47
  • In order to rewrite the history. git subtree combines the history at the root level, but I would like to have a history on every individual file. if you do git log repo1/pom.xml after subtree, there will be only 1 commit. I am sure that this is not the correct way of achieving what I want, but I haven't found the other way yet. – Solaris Jul 27 '23 at 13:55
  • Would this work for you? https://stackoverflow.com/questions/1425892/how-do-you-merge-two-git-repositories (see the note about subdirectory). This example seems relevant: https://github.com/newren/git-filter-repo/#simple-example-with-comparisons – chelmertz Jul 27 '23 at 20:30
  • Yes. This is where I started and what, in fact, I tried to use. It works only if I need to combine 2 repositories. If I try to combine more, I end up with the result from my question. When I try to do the same steps to add the second repository, the history for the first one vanishes. – Solaris Jul 28 '23 at 08:28

1 Answers1

0

Git submodules are the tool you are looking for.

Their usage is loved by some, hated by others, but they objectively do exactly what you want.

Documentation: Git - Submodules:

It often happens that while working on one project, you need to use another project from within it. Perhaps it’s a library that a third party developed or that you’re developing separately and using in multiple parent projects. A common issue arises in these scenarios: you want to be able to treat the two projects as separate yet still be able to use one from within the other.

YSC
  • 38,212
  • 9
  • 96
  • 149
  • 1
    As I know, submodules create a link to an external repository. It doesn't copy the data. I am trying to create a separate repository with all the data from other repositories inside, that will not depend on external repositories. – Solaris Jul 27 '23 at 12:28
  • Why does it matter? – YSC Jul 27 '23 at 14:01
  • 1
    In order to build it as one multi module project and get rid of multiple repositories that depend on each other. – Solaris Jul 27 '23 at 14:53