2

I am trying to find a way to clone all the git repositories from a remote GitLab group with Python (gitPython for example).

Can you help me, please?

Amir
  • 1,926
  • 3
  • 23
  • 40
mo45
  • 21
  • 2
  • A simple web search can give you same options: https://lmgtfy.app/?q=git+clone+all+repositories I don't think there's a single command to that, you'd probably need to list all the repos from the group, parse the response and then clone the repos. Something like: https://gist.github.com/milanboers/f34cdfc3e1ad9ba02ee8e44dae8e093f – acrespo Feb 09 '21 at 19:00
  • without gitPython , you can use the api https://docs.gitlab.com/ee/api/groups.html#list-a-groups-projects to list all the repos from a group – mo45 Feb 16 '21 at 09:16

1 Answers1

1

You should install python-gitlab (to iterate over the GitLab project structures) and GitPython (to clone the Git repos) packages first.

If you have a self-hosted GitLab instance, then you also need to get a personal token. Go to https://gitlab.your-company.com/-/profile/personal_access_tokens to access the token interface, like on the screenshot:

Personal Token

Save the obtained token (step 1).

You also need a deploy token for each repository that you are going to clone. To do so, go to https://gitlab.your-company.com/your_group/your_project/-/settings/repository -> "Deploy tokens", like on the screenshot below:

Deploy Token Example

You'll get a token username (like gitlab+deploy-token-1488) and the token itself. Save this username/token pair (step 2) and repeat this step for each repo.

As the last step, you should get the ID of the group which contains your repos. It should be displayed on the web interface of your group at https://gitlab.your-company.com/your_group like on the screenshot:

Group ID Example

Save this ID too (step 3).

Now you are all set to run this code snippet:

import git
import os
import gitlab

local_cloned_repos_dir = 'local_cloned_repos_dir'

gl = gitlab.Gitlab('https://gitlab.your-company.com', private_token='insert_token_from_step_1')

deploy_token_map = {
    'your_project_name': 'gitlab+deploy-token-1488:insert_token_from_step_2'
}

for project in gl.groups.get('insert_id_from_step_3').projects.list():
    local_cloned_project_dir = os.path.join(local_cloned_repos_dir, project.name)
    os.makedirs(local_cloned_project_dir, exist_ok=True)
    git.Repo.clone_from('https://{}@gitlab.your-company.com/{}'.format(deploy_token_map[project.name], project.path_with_namespace), local_cloned_project_dir)

Note that I've created a map to match the name of the project with the corresponding token username / deploy token pair, but you don't have to handle it exactly this way.

Hope it helps!

Amir
  • 1,926
  • 3
  • 23
  • 40