1

I'm not very proficient in git beyond the usual commit/pull/push/branch etc. commands, so I don't know the proper words I need to search for what I want to do.

I'm looking for advice on how to get to what I want and pointers to URLs and such that give more explanation.

Over time a project has evolved from a single product to several subproducts that are used to build the final product. Each subproduct has it's own subdirectory tree but the Jenkins jobs that build the subproducts and the final product still pull in the entire repository just to build the one product it's supposed to do.

This takes quite some build time, even for the small products that don't change a lot over time. So I'm thinking of splitting off each of the projects into its own repository. At the same time they all follow a similar pattern for several files so I'd like to move that into common code that is used by all. Here is a schematic layout

/project
  /subproject1
    /common
    /src
    /test
  /subproject2
    /common
    /src
    /test
  /common # main project
  /src    # main project
  /test   # main project

From what I've read so far I think the best thing to do is to split all subprojects and the main project into separate repositories. Preserving history would be nice, but commits have spanned multiple subprojects in the past and no branches were used, so that might not be feasible.

Also, when I move the common code off into its own repository, how do I get the files back into the common directory so that a change in the common code gets pushed to its own repository? I've read about git-subtree that could do the trick, but I've never used it.

Is this the best way forward or are there simpler alternatives?

CodeWizard
  • 128,036
  • 21
  • 144
  • 167
hepabolu
  • 1,214
  • 4
  • 18
  • 29
  • Possible duplicate of http://stackoverflow.com/questions/359424/detach-subdirectory-into-separate-git-repository/17864475 – J.J. Hakala Jan 17 '16 at 09:31
  • @J.J.Hakala That question is good, but since it's some years old, I wondered if ideas have changed since then. Also, I'm not sure if I should use `subtree` or `filter-branch` because we didn't exactly follow any gitflow (aka none). I would very much read some advice on which way to go. – hepabolu Jan 17 '16 at 13:35
  • 1
    One of the answers, "The Easy Way™" is more recent (2013), http://stackoverflow.com/questions/359424/detach-subdirectory-into-separate-git-repository/17864475#17864475 – J.J. Hakala Jan 17 '16 at 18:25
  • @J.J.Hakala Thanks. The explanation using subtree helped in splitting the repository in separate projects based on a subdirectory worked as step one. – hepabolu Jan 23 '16 at 12:30

1 Answers1

0

Over time a project has evolved from a single product to several subproducts that are used to build the final product.
...
So I'm thinking of splitting off each of the projects into its own repository

You should submodules.

enter image description here

Submodules is a independent project inside another project, which is self managed in its own folder.

In your case since you already have the code inside a big one repository so you will have to break it down to small pieces.

To do it use this folder bfc. In your case simply clean out the subfolders which you are changing to be submodules.

enter image description here

CodeWizard
  • 128,036
  • 21
  • 144
  • 167
  • 1
    Thanks. I didn't know about BFG. I've used it to remove binary blobs like Excel spreadsheets and Word documents from the repository and shrink the size that way. – hepabolu Jan 23 '16 at 12:31
  • 1
    I would like to accept your answer, but it didn't solve all of my questions. @J.J.Hakala also set me in the right direction. I'm not sure what is the correct way to give credit to both. – hepabolu Jan 24 '16 at 11:54
  • I've split the big repo using `git subtree --prefix [dir] -b newbranch` as described in the post @J.J.Hakala pointed to. From there I cloned each branch into it's own repository and used your submodules suggestion to add the common repo to each of the stand-alone ones. If proper behavior here requires a new answer to explain, please let me know. – hepabolu Jan 25 '16 at 19:34
  • Same problem, different repo. How can I scrub parts of files that contain sensitive info from the git history? – hepabolu Jan 25 '16 at 19:36
  • Again you can use this method as well. – CodeWizard Jan 25 '16 at 20:20