0

I have a software project foo on an internal GitLab repository hosted by my company and want to partially publish it as an open source project baa on GitHub.

Lets say I put the public part in a folder "public":

foo/public

and the private parts in a folder "private":

foo/private

How can I sync the public folder with a public GitHub repository https://github.com/user/new_repository.git ?

In my wildest dreams I would have:

a) An automation pipeline (gitlab-ci.yml) in foo, forwarding/syncing each change of the public folder to baa

b) Allow external users to contribute to baa and be somehow able to merge their changes back to foo.

c) As an alternative to a) have a way to sync only from time to time, after certain requirements are fulfilled (e.g. export control of a new feature).

I saw the git commands

  1. git submodule

  2. git subtree

and the

  1. repository mirroring feature of GitLab:

https://docs.gitlab.com/ee/user/project/repository/mirror/

but I am not sure if they are applicable for my use case.

Let' say I would like to use parts of foo as submodule in baa... how to ensure, that only the folder public is considered instead of the whole project?

Maybe my approach is wrong and I should first try to separate the public and private parts to individual projects.

Related questions and articles:

How to link folder from a git repo to another repo?

How to mirror a GitLab repository to GitHub using SSH keys?

https://www.atlassian.com/de/git/tutorials/git-subtree

How do I git "subtree split" but maintain the full folder hierarchy?

Stefan
  • 10,010
  • 7
  • 61
  • 117
  • do you need the history on the public repository or you can start from now ? – Ôrel Jun 12 '23 at 09:13
  • I don't need the history. – Stefan Jun 12 '23 at 10:36
  • 1
    create a new repository with the public content, delete the public content and add the newly created repository as a submodule – Ôrel Jun 12 '23 at 10:57
  • 1
    See also https://stackoverflow.com/questions/359424/detach-move-subdirectory-into-separate-git-repository – mkrieger1 Jun 13 '23 at 06:45
  • How to add the content of the new submodule while keeping the original folder structure? https://stackoverflow.com/questions/76462811/how-to-add-the-content-of-a-git-submodule-directly-in-the-curent-directory-with – Stefan Jun 13 '23 at 08:00

1 Answers1

0

With the help of the comments I found the solutions shown below.

  • A uses submodule and is a closer match to a) from the question.

  • B uses subtree and is a closer match to c) from the question and seems to be less complicated.

0. Fill a separate repository with content of public folder

Push content of folder "public" to new repository:

cd public    
git init --initial-branch=main    
git remote add origin https://github.com/user/new_repository.git    
git add .    
git commit -m "Initial commit"     
git push -u origin main

A. Use separate repository as submodule "public"

Integrate submodule

  • Delete public folder inside main folder

  • Commit main folder (=> apply deletion of public folder)

  • Add new repository as a submodule "public":

    git submodule add -b main https://github.com/user/new_repository.git public

    git submodule update --remote

The git submodule add command creates a file .gitmodules containing

[submodule "public"]
    path = public
    url = https://github.com/user/new_repository.git
    branch = main
  • Commit and push main folder

Fresh Clone

If someone else does a fresh clone of the main project, the recursive option needs to be used. Otherwise the content of the submodule folder "public" will be empty.

enter image description here

Commit change of submodule

If someone changes the content of the submodule, the submodule and the main project need to be committed/pushed separately. This seems to be a disadvantage of the submodule approach.

If I change a file of the submodule, the main project does not recognize that as a change (list of commit dialog will be empty). Once I committed the change using the submodule folder, the main project does recognize a change (new version of submodule). Then the main project needs to be committed, too.

The push dialog of TortoiseGit has the value "On demand" for the "Recurse submodule" option:

enter image description here

Therefore, at least the push action can be done for both, the main and the submodule project. (If there is a single command to commit and push both, please let me know.)

Related questions on usage of submodules

How do I "git clone" a repo, including its submodules?

How do I "commit" changes in a git submodule?

How to add the content of a git submodule directly in the curent directory, without extra project folder?

git submodule tracking latest

B. Use separate repository as subtree "public"

Include subtree

  • Delete public folder inside main folder

  • Commit main folder (=> apply deletion of public folder)

  • Use subtree command to include files of repository locally

git subtree add --prefix public https://github.com/user/new_repository.git main --squash
  • Push main folder (commit is already done by subtree add command)

Fresh Clone

  • A fresh clone includes all files of the subtree. In comparison to submodules no extra command or option is required.

Commit change of subtree

  • A change in the subtree is recognized by the parent/main project as normal file change and is committed with the main project by default.

  • When committing to the main project, the repository for the subtree is not automatically updated.

  • In order to apply the local changes to the subtree repository, use following explicit command:

git subtree push --prefix public https://github.com/user/new_repository.git main --squash

Its possible to introduce an alias for the url, so that the over all length of the command gets shorter, using git add remote, see https://www.atlassian.com/git/tutorials/git-subtree

Pull changes of subtree

git subtree pull --prefix public https://github.com/user/new_repository.git main

Related articles on subtree

Stefan
  • 10,010
  • 7
  • 61
  • 117