In my GitLab repository, I have a group with 20 projects. I want to clone all projects at once. Is that possible?
30 Answers
One liner with curl
, jq
, tr
:
for repo in $(curl -s --header "PRIVATE-TOKEN: your_private_token" https://<your-host>/api/v4/groups/<group_id> | jq -r ".projects[].ssh_url_to_repo"); do git clone $repo; done;
For Gitlab.com use https://gitlab.com/api/v4/groups/<group_id>
To include subgroups add include_subgroups=true
query param like
https://<your-host>/api/v4/groups/<group_id>?include_subgroups=true
Note: To clone with http url use
http_url_to_repo
instead ofssh_url_to_repo
in jq (Thanks @MattVon for the comment)

- 20,532
- 7
- 64
- 57
-
5Nice and simple! Best one-off answer so far. Worth mentioning that this works only in `bash` and `curl` and `jq` must be installed. – Nader Ghanbari Jan 06 '20 at 06:16
-
This works only if know the group id and there are not any subgroups. – SaTa Feb 02 '20 at 14:16
-
1Okay, one can use `include_subgroups=true` to get all the projects. – SaTa Feb 02 '20 at 20:29
-
I had to add quotes to the jq part. `for repo in $(curl "https://
/api/v4/groups/ – Tom May 14 '20 at 13:46?private_token= " | 'jq .projects[].ssh_url_to_repo' | tr -d '"'); do git clone $repo; done;` -
3this only gives the projects which are in the group folder. How do I get the projects which are housed in the subgroups (subfolders)? – skrisshnaswamy Jun 16 '20 at 05:00
-
May be u can include `include_subgroups=true` @skrisshnaswamy – Dinesh Balasubramanian Jun 19 '20 at 12:49
-
@DineshBalasubramanian where do we include subgroup? can you post full command with `include_subgroups=true` – uberrebu Apr 30 '21 at 02:03
-
4Didn't work for me for subgroups, even after adding `include_subgroups=true`. However, the answer by @Ruben works fine. The API URL needs to have `/projects` in it. – rbawaskar Aug 04 '21 at 18:01
-
just a quick note here, I managed to do it with a replace on "PRIVATE-TOKEN" with "remember_user_token". ps: have no idea if the original script works – CallMeLoki Aug 09 '21 at 09:05
-
7using the `include_subgroups=true` didn’t work for me. So I used the following which works fine: `for repo in $(curl -s --header "PRIVATE-TOKEN:
" https://gitlab.com/api/v4/groups/ – ehsan khodadadi Jan 26 '22 at 18:18/projects\?include_subgroups\=true | jq ".[].ssh_url_to_repo" | tr -d '"'); do git clone $repo; done;` -
The solution as is will output only 100 projects of the group. To get all projects API URL needs to have `/projects` in it with subsequent iteration over pages https://docs.gitlab.com/ee/api/index.html#pagination. So to get all projects one more inner loop is required for pagination. – ks1322 May 30 '22 at 11:23
-
1nice ! work perfect. small contrib. : To clone with http url use : `http_url_to_repo` instead of `ssh_url_to_repo` in jq request – MattVon Jun 24 '22 at 06:40
-
1Some additional info could be more helpful in the answer. For example, to find `PRIVATE TOKEN` - we will have to generate it from the `Gitlab user settings -> Access tokens -> Personal access tokens`. I also had to add my SSH public key to the Gitlab account under `Gitlab user settings -> SSH keys`. – Reaz Murshed Jun 27 '22 at 22:52
-
I'm trying to get just the projects from this using jq, so that I can then pipe and filter by projects where is_empty: false and is_archived: false. Any pointers? – MrMesees Sep 07 '22 at 12:40
-
This answer, i-ncluding the additions in the comments (e.g. `/projects`)- does not work correctly and the directory structure from gitlab is lost. – DimiDak Jul 20 '23 at 16:29
Update Dec. 2022, use glab repo clone
glab repo clone -g <group> -p --paginate
With:
-p
,--preserve-namespace
: Clone the repo in a subdirectory based on namespace--paginate
: Make additional HTTP requests to fetch all pages of projects before cloning. Respects--per-page
That does support cloning more than 100 repositories (since MR 1030, and glab v1.24.0, Dec. 2022)
This is for gitlab.com
or for a self-managed GitLab instance, provided you set the environment variable GITLAB_URI
or GITLAB_HOST
: it specifies the URL of the GitLab server if self-managed (eg: https://gitlab.example.com
).
Original answer and updates (starting March 2015):
Not really, unless:
you have a 21st project which references the other 20 as submodules.
(in which case a clone followed by agit submodule update --init
would be enough to get all 20 projects cloned and checked out)or you somehow list the projects you have access (GitLab API for projects), and loop on that result to clone each one (meaning that can be scripted, and then executed as "one" command)
Since 2015, Jay Gabez mentions in the comments (August 2019) the tool gabrie30/ghorg
ghorg
allows you to quickly clone all of an org's or user's repos into a single directory.
Usage:
$ ghorg clone someorg
$ ghorg clone someuser --clone-type=user --protocol=ssh --branch=develop
$ ghorg clone gitlab-org --scm=gitlab --namespace=gitlab-org/security-products
$ ghorg clone --help
Also (2020): https://github.com/ezbz/gitlabber
usage: gitlabber [-h] [-t token] [-u url] [--debug] [-p]
[--print-format {json,yaml,tree}] [-i csv] [-x csv]
[--version]
[dest]
Gitlabber - clones or pulls entire groups/projects tree from gitlab

- 1,262,500
- 529
- 4,410
- 5,250
-
9[ghorg](https://github.com/gabrie30/ghorg) is a small cli that will do this for you – jimjam Aug 03 '19 at 15:17
-
2@JayGabez Thank you. I have included your comment in the answer for more visibility. – VonC Aug 03 '19 at 15:22
-
If this is a feature your entire team would be using often, I'd recommend automating the creation of a collection repository (as per the first suggestion). This would keep everything nice and consistent, and avoid the need to make your developers use additional tools besides a vanilla git client of their choice. It would also be fairly simple to achieve through the API and webhooks. Of course, be wary of unintended feedback loops... – Samuel Willems Aug 03 '19 at 15:37
-
2
-
1@EricBDev The [README](https://gitlab.com/gitlab-org/cli#user-content-glab) does mention "`glab` is available for repositories hosted on GitLab.com and self-managed GitLab instances". See "[Authentication](https://gitlab.com/gitlab-org/cli#authentication)". And "[Environment variables](https://gitlab.com/gitlab-org/cli#environment-variables)": "`GITLAB_URI` or `GITLAB_HOST`: specify the URL of the GitLab server if self-managed (eg: `https://gitlab.example.com`). Default is `https://gitlab.com`." – VonC Jan 18 '23 at 15:06
-
1ok, finally got it running: I fought a bit to install glab under ubuntu22 in WSL2 and then had to set the GITLAB_URI and GITLAB_TOKEN variables for glab to run without 401. – EricBDev Jan 18 '23 at 16:39
-
@EricBDev Well done! That should work indeed. Note that I use glab directly on Windows, without needing WSL2. – VonC Jan 18 '23 at 17:34
-
@VonC Does glab update (git pull) the repos after the initial cloning? i.e. Can glab keep the local repos in sync with the upstream? – rootkea Apr 23 '23 at 08:35
-
@rootkea A clone always entails a pull for the default branch of the cloned repository. But after that, there is no "automatic sync". – VonC Apr 23 '23 at 18:36
-
@VonC Okay. So that's unlike `gitlabber` and `ghorg` which git pull the existing repos and clone the new ones keeping the local copies in sync with the remote. – rootkea Apr 24 '23 at 00:34
-
@rootkea True. It is however similar to the [GitHub CLI `gh`](https://cli.github.com/). – VonC Apr 24 '23 at 05:25
Here's an example in Python 3:
from urllib.request import urlopen
import json
import subprocess, shlex
allProjects = urlopen("https://[yourServer:port]/api/v4/projects?private_token=[yourPrivateTokenFromUserProfile]&per_page=100000")
allProjectsDict = json.loads(allProjects.read().decode())
for thisProject in allProjectsDict:
try:
thisProjectURL = thisProject['ssh_url_to_repo']
command = shlex.split('git clone %s' % thisProjectURL)
resultCode = subprocess.Popen(command)
except Exception as e:
print("Error on %s: %s" % (thisProjectURL, e.strerror))

- 25,404
- 19
- 49
- 81

- 671
- 6
- 4
-
1I had some issues deserializing the json due to unknown encoding issues, in this case a modification to parse by regular expression helped: `urls = re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+.git', allProjects.read().decode("latin-1")) ` – 79E09796 Mar 02 '16 at 17:06
-
The script works fine. I had two issues: http://stackoverflow.com/a/31601343/2777965 and I had to use https:// instead of http – 030 Feb 13 '17 at 21:17
-
This script uses the old api version, a current url would be https://[yourServer:port]/api/v4/projects?private_token=[yourPrivateTokenFromUserProfile]&per_page=100 – aseques Dec 12 '17 at 09:22
-
1
There is a tool called myrepos, which manages multiple version controls repositories. Updating all repositories simply requires one command:
mr update
In order to register all gitlab projects to mr, here is a small python script. It requires the package python-gitlab installed:
import os
from subprocess import call
from gitlab import Gitlab
# Register a connection to a gitlab instance, using its URL and a user private token
gl = Gitlab('http://192.168.123.107', 'JVNSESs8EwWRx5yDxM5q')
groupsToSkip = ['aGroupYouDontWantToBeAdded']
gl.auth() # Connect to get the current user
gitBasePathRelative = "git/"
gitBasePathRelativeAbsolut = os.path.expanduser("~/" + gitBasePathRelative)
os.makedirs(gitBasePathRelativeAbsolut,exist_ok=True)
for p in gl.Project():
if not any(p.namespace.path in s for s in groupsToSkip):
pathToFolder = gitBasePathRelative + p.namespace.name + "/" + p.name
commandArray = ["mr", "config", pathToFolder, "checkout=git clone '" + p.ssh_url_to_repo + "' '" + p.name + "'"]
call(commandArray)
os.chdir(gitBasePathRelativeAbsolut)
call(["mr", "update"])

- 91
- 1
- 3
-
Here's an updated version (gl.Project doesn't exist as API, now is gl.proejcts.list) and a python 2 version: https://gist.github.com/maxgalbu/995a42a9a4e8594b4a628df93985fc2f – maxgalbu Aug 08 '17 at 10:50
-
I built a script (curl, git, jq required) just for that. We use it and it works just fine: https://gist.github.com/JonasGroeger/1b5155e461036b557d0fb4b3307e1e75
To find out your namespace, its best to check the API quick:
curl "https://domain.com/api/v3/projects?private_token=$GITLAB_PRIVATE_TOKEN"
There, use "namespace.name" as NAMESPACE
for your group.
The script essentially does:
- Get all Projects that match your
PROJECT_SEARCH_PARAM
Get their
path
andssh_url_to_repo
2.1. If the directory
path
exists, cd into it and callgit pull
2.2. If the directory
path
does not exist, callgit clone

- 114
- 1
- 6

- 1,558
- 2
- 21
- 35
-
1
-
Hello @KosratD.Ahmad! You might want to explore the GitLab API for that. It's possible but I don't have a use case to develop the feature. – Jonas Gröger Jun 10 '18 at 10:00
-
@KosratD.Ahmad - it seems I can't @ you from another answer - see my answer and comment (f you're still active) – bob dylan Nov 25 '19 at 11:44
-
v3 api is depricated use v4 instead : {"error":"API V3 is no longer supported. Use API V4 instead."} – Med Ali Difallah Nov 08 '21 at 09:30
Lot of good answers, but here's my take. Use it if you:
- want to clone everything in parallel
- have your ssh keys configured to clone from the server without entering a password
- don't want to bother creating an access token
- are using a limited shell like git bash (without
jq
)
So, using your browser, acess https://gitlab.<gitlabserver>/api/v4/groups/<group name>?per_page=1000
download the json with all projects info and save it as a file named group.json
.
Now just run this simple command in the same dir:
egrep -o 'git@[^"]+.git' group.json|xargs -n 1 -P 8 git clone
Increase the number in -P 8
to change the number of parallel processes. If you have more than a thousand repositories, increase the number after the perpage=
.
If <group name>
has spaces or accented chars, note that it must be url encoded.
If you want to automate the download, the easiest way to authenticate is to generate a access token in GitLab/GitHub and put it in the url: https://user:access_toke@mygitlab.net/api/v4/groups/<group name>?per_page=1000
.
@jocullin gave a nice tip in the comments to also download subgroups. Just add &include_subgroups=true
to the groups url above.

- 33,186
- 27
- 159
- 192
-
1My loved method, but I prefer [jq](https://stedolan.github.io/jq/manual/). However, this solution downloads no more than 20 projects by default. Specify `per_page=100` and `page` query parameters to retrieve more. – mymedia Jun 01 '21 at 15:03
-
2This approach made it simple to export a set of projects. If you have subgroups, i.e. parent/child, the `
` will be the url encoded value, i.e. `parent%2Fchild` – MarkBarry Dec 23 '21 at 00:15 -
This works! I had no idea that even private group/projects where publicly queryable... I just tested, they are. – melMass Aug 10 '22 at 12:41
-
1Thanks for this! Here's a variation on the first url to get all projects, even in sub-groups. `https://gitlab.com/api/v4/groups/top_level_group_name/projects?per_page=1000&include_subgroups=true` – joecullin Jun 26 '23 at 19:43
Here is another example of a bash script to clone all the repos in a group. The only dependency you need to install is jq (https://stedolan.github.io/jq/). Simply place the script into the directory you want to clone your projects into. Then run it as follows:
./myscript <group name> <private token> <gitlab url>
i.e.
./myscript group1 abc123tyn234 http://yourserver.git.com
Script:
#!/bin/bash
if command -v jq >/dev/null 2>&1; then
echo "jq parser found";
else
echo "this script requires the 'jq' json parser (https://stedolan.github.io/jq/).";
exit 1;
fi
if [ -z "$1" ]
then
echo "a group name arg is required"
exit 1;
fi
if [ -z "$2" ]
then
echo "an auth token arg is required. See $3/profile/account"
exit 1;
fi
if [ -z "$3" ]
then
echo "a gitlab URL is required."
exit 1;
fi
TOKEN="$2";
URL="$3/api/v3"
PREFIX="ssh_url_to_repo";
echo "Cloning all git projects in group $1";
GROUP_ID=$(curl --header "PRIVATE-TOKEN: $TOKEN" $URL/groups?search=$1 | jq '.[].id')
echo "group id was $GROUP_ID";
curl --header "PRIVATE-TOKEN: $TOKEN" $URL/groups/$GROUP_ID/projects?per_page=100 | jq --arg p "$PREFIX" '.[] | .[$p]' | xargs -L1 git clone

- 1,371
- 1
- 20
- 47
-
This worked for me from git bash shell on windows 7 perfectly. The jq-win64.exe file downloaded had to be renamed to jq.exe and placed in a folder in the search path (for me ~/bin/jq.exe). IN gitlab I had to generate a personal access token and make sure to store it somewhere safe for future uses. – Farrukh Najmi May 29 '18 at 13:36
Yep it's possible, here is the code.
prerequisites:
pip install python-gitlab
#!/usr/bin/python3
import os
import sys
import gitlab
import subprocess
glab = gitlab.Gitlab(f'https://{sys.argv[1]}', f'{sys.argv[3]}')
groups = glab.groups.list()
groupname = sys.argv[2]
for group in groups:
if group.name == groupname:
projects = group.projects.list(all=True)
for repo in projects:
command = f'git clone {repo.ssh_url_to_repo}'
process = subprocess.Popen(command, stdout=subprocess.PIPE, shell=True)
output, _ = process.communicate()
process.wait()
Example:
- create .py file (ex. gitlab-downloader.py)
- copy-paste code from above
- on Linux OS (or OSX) do chmod +x on the script file (ex. chmod +x gitlab-downloader.py)
- run it with 3 params: Gitlab hostname, groupname, your Personal Access Token(see https://gitlab.exmaple.com/profile/personal_access_tokens)

- 71
- 1
- 1
I created a tool for that: https://github.com/ezbz/gitlabber, you can use glob/regex expressions to select groups/subgroups you'd like to clone.
Say your top-level group is called MyGroup
and you want to clone all projects under it to ~/GitlabRoot
you can use the following command:
gitlabber -t <personal access token> -u <gitlab url> -i '/MyGroup**' ~/GitlabRoot

- 79
- 1
- 4
-
Although it seemed to be scanning everything at the end I got this: `gitlabber.cli - CRITICAL - The tree is empty, check your include/exclude patterns or run with more verbosity for debugging` – DimiDak Jul 20 '23 at 16:43
If you are okay with some shell sorcery this will clone all the repos grouped by their group-id (you need jq and parallel)
seq 3 \
| parallel curl -s "'https://[gitlabUrl]/api/v4/projects?page={}&per_page=100&private_token=[privateToken]'
| jq '.[] | .ssh_url_to_repo, .name, .namespace.path'" \
| tr -d '"' \
| awk '{ printf "%s ", $0; if (NR % 3 == 0) print " " }' \
| parallel --colsep ' ' 'mkdir -p {2} && git clone {1} {3}/{2}'

- 453
- 6
- 8
Using curl, jq and tr and the same approach described previously, but for more than 20 projects:
for repo in $(curl --header "PRIVATE-TOKEN:<Private-Token>" -s "https://<your-host>/api/v4/groups/<group-id>/projects?include_subgroups=true&per_page=100&page=n" | jq '.[].ssh_url_to_repo' | tr -d '"'); do git clone $repo; done;
For Gitlab.com use https://gitlab.com/api/v4/groups/[group-id]/projects
Only need to iterate changing page number.

- 61
- 1
- 1
An updated Python 3 script that accomplishes this really effectively using Gitlab's latest api and proper pagination:
import requests
import subprocess, shlex
import os
print('Starting getrepos process..')
key = '12345678901234567890' # your gitlab key
base_url = 'https://your.gitlab.url/api/v4/projects?simple=true&per_page=10&private_token='
url = base_url + key
base_dir = os.getcwd()
while True:
print('\n\nRetrieving from ' + url)
response = requests.get(url, verify = False)
projects = response.json()
for project in projects:
project_name = project['name']
project_path = project['namespace']['full_path']
project_url = project['ssh_url_to_repo']
os.chdir(base_dir)
print('\nProcessing %s...' % project_name)
try:
print('Moving into directory: %s' % project_path)
os.makedirs(project_path, exist_ok = True)
os.chdir(project_path)
cmd = shlex.split('git clone --mirror %s' % project_url)
subprocess.run(cmd)
except Exception as e:
print('Error: ' + e.strerror)
if 'next' not in response.links:
break
url = response.links['next']['url'].replace('127.0.0.1:9999', 'your.gitlab.url')
print('\nDone')
Requires the requests library (for navigating to the page links).

- 1,339
- 1
- 24
- 29
-
It is not downloading the actual project. Just creating some junk folder inside the project like this app1.git, app2.git – Vikas Rathore Aug 21 '19 at 07:42
-
-
Oh, well, right – if you're wanting to actually check out the head commits into working branches then you need to remove `--mirror` because it implies `--bare`. However, I would say that this is probably not what you want to do.. – forresthopkinsa Feb 04 '23 at 02:52
I have written the script to pull the complete code base from gitlab for particular group.
for pag in {1..3} // number of pages projects has span {per page 20 projects so if you have 50 projects loop should be 1..3}
do
curl -s http://gitlink/api/v4/groups/{groupName}/projects?page=$pag > url.txt
grep -o '"ssh_url_to_repo": *"[^"]*"' url.txt | grep -o '"[^"]*"$' | while read -r line ; do
l1=${line%?}
l2=${l1:1}
echo "$l2"
git clone $l2
done
done

- 152
- 1
- 4
- 10
Here's a Java version that worked for me using gitlab4j with an access token and git command.
I ran this on Windows and Mac and it works. For Windows, just add 'cmd /c' before 'git clone' inside the .exec()
void doClone() throws Exception {
try (GitLabApi gitLabApi = new GitLabApi("[your-git-host].com/", "[your-access-token]");) {
List<Project> projects = gitLabApi.getGroupApi().getProjects("[your-group-name]");
projects.forEach(p -> {
try {
Runtime.getRuntime().exec("git clone " + p.getSshUrlToRepo(), null, new File("[path-to-folder-to-clone-projects-to]"));
} catch (Exception e) {
e.printStackTrace();
}
});
}
}

- 41
- 3
I know this question is a few years old, but I had problems with awk/sed and the scripts here (macOS). I wanted to clone a root-group including their subgroups while keeping the tree structure.
My python script:
#!/usr/bin/env python3
import os
import re
import requests
import posixpath
import argparse
from git import Repo
parser = argparse.ArgumentParser('gitlab-clone-group.py')
parser.add_argument('group_id', help='id of group to clone (including subgroups)')
parser.add_argument('directory', help='directory to clone repos into')
parser.add_argument('--token', help='Gitlab private access token with read_api and read_repository rights')
parser.add_argument('--gitlab-domain', help='Domain of Gitlab instance to use, defaults to: gitlab.com', default='gitlab.com')
args = parser.parse_args()
api_url = 'https://' + posixpath.join(args.gitlab_domain, 'api/v4/groups/', args.group_id, 'projects') + '?per_page=9999&page=1&include_subgroups=true'
headers = {'PRIVATE-TOKEN': args.token}
res = requests.get(api_url, headers=headers)
projects = res.json()
base_ns = os.path.commonprefix([p['namespace']['full_path'] for p in projects])
print('Found %d projects in: %s' % (len(projects), base_ns))
abs_dir = os.path.abspath(args.directory)
os.makedirs(abs_dir,exist_ok=True)
def get_rel_path(path):
subpath = path[len(base_ns):]
if (subpath.startswith('/')):
subpath = subpath[1:]
return posixpath.join(args.directory, subpath)
for p in projects:
clone_dir = get_rel_path(p['namespace']['full_path'])
project_path = get_rel_path(p['path_with_namespace'])
print('Cloning project: %s' % project_path)
if os.path.exists(project_path):
print("\tProject folder already exists, skipping")
else:
print("\tGit url: %s" % p['ssh_url_to_repo'])
os.makedirs(clone_dir, exist_ok=True)
Repo.clone_from(p['ssh_url_to_repo'], project_path)
Usage
- Download the gitlab-clone-group.py
- Generate a private access token with read_api and read_repository rights
- Get your group ID (displayed in light gray on below your group name)
- Run the script
Example:
python3 gitlab-clone-group.py --token glabc-D-e-llaaabbbbcccccdd 12345678 .
Clones the group 12345678 (and subgroups) into the current working directory, keeping the tree structure.
Help:
usage: gitlab-clone-group.py [-h] [--token TOKEN] [--gitlab-domain GITLAB_DOMAIN] group_id directory
positional arguments:
group_id id of group to clone (including subgroups)
directory directory to clone repos into
options:
-h, --help show this help message and exit
--token TOKEN Gitlab private access token with read_api and read_repository rights
--gitlab-domain GITLAB_DOMAIN
Domain of Gitlab instance to use, defaults to: gitlab.com

- 758
- 1
- 7
- 17
You can refer to this ruby script here: https://gist.github.com/thegauraw/da2a3429f19f603cf1c9b3b09553728b
But you need to make sure that you have the link to the organization gitlab url (which looks like: https://gitlab.example.com/api/v3/ for example organization) and private token (which looks like: QALWKQFAGZDWQYDGHADS and you can get in: https://gitlab.example.com/profile/account once you are logged in). Also do make sure that you have httparty gem installed or gem install httparty

- 5,428
- 2
- 21
- 14
Based on this answer, with personal access token instead of SSH to git clone.
One liner with curl
, jq
, tr
Without subgroups :
for repo in $(curl -s --header "PRIVATE-TOKEN: <private_token>" https://<your-host>/api/v4/groups/<group-name> | jq ".projects[]".http_url_to_repo | tr -d '"' | cut -c 9-); do git clone https://token:<private_token>@$repo; done;
Including subgroups :
for repo in $(curl -s --header "PRIVATE-TOKEN: <private_token>" "https://<your-host>/api/v4/groups/<group-name>/projects?include_subgroups=true&per_page=1000" | jq ".[]".http_url_to_repo | tr -d '"' | cut -c 9-); do git clone https://token:<private_token>@$repo; done;
Please note that the private_token
for the curl must have API
rights. The private_token
for the git clone
must have at least read_repository
rights. It can be the same token (if it has API
rights), but could also be 2 differents tokens

- 652
- 4
- 15
Another way to do it with Windows "Git Bash" that has limited packages installed :
#!/bin/bash
curl -o projects.json https://<GitLabUrl>/api/v4/projects?private_token=<YourToken>
i=0
while : ; do
echo "/$i/namespace/full_path" > jsonpointer
path=$(jsonpointer -f jsonpointer projects.json 2>/dev/null | tr -d '"')
[ -z "$path" ] && break
echo $path
if [ "${path%%/*}" == "<YourProject>" ]; then
[ ! -d "${path#*/}" ] && mkdir -p "${path#*/}"
echo "/$i/ssh_url_to_repo" > jsonpointer
url=$(jsonpointer -f jsonpointer projects.json 2>/dev/null | tr -d '"')
( cd "${path#*/}" ; git clone --mirror "$url" )
fi
let i+=1
done
rm -f projects.json jsonpointer

- 106
- 5
In response to @Kosrat D. Ahmad as I had the same issue (with nested subgroups - mine actually went as much as 5 deep!)
#!/bin/bash
URL="https://mygitlaburl/api/v4"
TOKEN="mytoken"
function check_subgroup {
echo "checking $gid"
if [[ $(curl --header "PRIVATE-TOKEN: $TOKEN" $URL/groups/$gid/subgroups/ | jq .[].id -r) != "" ]]; then
for gid in $(curl --header "PRIVATE-TOKEN: $TOKEN" $URL/groups/$gid/subgroups/ | jq .[].id -r)
do
check_subgroup
done
else
echo $gid >> top_level
fi
}
> top_level #empty file
> repos #empty file
for gid in $(curl --header "PRIVATE-TOKEN: $TOKEN" $URL/groups/ | jq .[].id -r)
do
check_subgroup
done
# This is necessary because there will be duplicates if each group has multiple nested groups. I'm sure there's a more elegant way to do this though!
for gid in $(sort top_level | uniq)
do
curl --header "PRIVATE-TOKEN: $TOKEN" $URL/groups/$gid | jq .projects[].http_url_to_repo -r >> repos
done
while read repo; do
git clone $repo
done <repos
rm top_level
rm repos
Note: I use jq .projects[].http_url_to_repo this can be replaced with .ssh_url_to_repo if you'd prefer.
Alternatively strip out the rm's and look at the files individually to check the output etc.
Admittedly this will clone everything, but you can tweak it however you want.
Resources: https://docs.gitlab.com/ee/api/groups.html#list-a-groups-subgroups

- 1,458
- 1
- 14
- 32
-
@Kosrat D. Ahmad I know it's late - it'd be interesting to know what solution you came up with instead. – bob dylan Nov 25 '19 at 11:41
This is a bit improved version of the oneliner in @ruben-lohaus post.
- it will work for up to 100 repos in the group.
- will clone every repository in the group including the path.
requirements:
- grep
- jq
- curl
GITLAB_URL="https://gitlab.mydomain.local/api/v4/groups/1141/projects?include_subgroups=true&per_page=100&page=0"
GITLAB_TOKEN="ABCDEFABCDef_5n"
REPOS=$(curl --header "PRIVATE-TOKEN:${GITLAB_TOKEN}" -s "${GITLAB_URL}" | jq -r '.[].ssh_url_to_repo')
for repo in $(echo -e "$REPOS")
do git clone $repo $(echo $repo | grep -oP '(?<=:).*(?=.git$)')
done

- 149
- 5
An alternative based on Dmitriy's answer -- in the case you were to clone repositories in a whole group tree recursively.
#!/usr/bin/python3
import os
import sys
import gitlab
import subprocess
glab = gitlab.Gitlab(f'https://{sys.argv[1]}', f'{sys.argv[3]}')
groups = glab.groups.list()
root = sys.argv[2]
def visit(group):
name = group.name
real_group = glab.groups.get(group.id)
os.mkdir(name)
os.chdir(name)
clone(real_group.projects.list(all=True))
for child in real_group.subgroups.list():
visit(child)
os.chdir("../")
def clone(projects):
for repo in projects:
command = f'git clone {repo.ssh_url_to_repo}'
process = subprocess.Popen(command, stdout=subprocess.PIPE, shell=True)
output, _ = process.communicate()
process.wait()
glab = gitlab.Gitlab(f'https://{sys.argv[1]}', f'{sys.argv[3]}')
groups = glab.groups.list()
root = sys.argv[2]
for group in groups:
if group.name == root:
visit(group)

- 54
- 2
Modified @Hot Diggity's answer.
import json
import subprocess, shlex
allProjects = urlopen("https://gitlab.com/api/v4/projects?private_token=token&membership=true&per_page=1000")
allProjectsDict = json.loads(allProjects.read().decode())
for thisProject in allProjectsDict:
try:
thisProjectURL = thisProject['ssh_url_to_repo']
path = thisProject['path_with_namespace'].replace('/', '-')
command = shlex.split('git clone %s %s' % (thisProjectURL, path))
p = subprocess.Popen(command)
p_status = p.wait()
except Exception as e:
print("Error on %s: %s" % (thisProjectURL, e.strerror))

- 2,522
- 2
- 18
- 23
For powershell (replace and and pass in a private token from gitlab (or hardcode it)):
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$url="https://<gitlab host>/api/v4/groups/<group>/projects?
simple=1&include_subgroups=true&private_token="+$args[0]
$req = Invoke-WebRequest $url | ConvertFrom-Json
foreach( $project in $req ) {
Start-Process git -ArgumentList "clone", $project.ssh_url_to_repo
}

- 548
- 3
- 13
One liner python3 version of Dinesh Balasubramanian response.
I only made this for lack of jq, only python3 (requests)
import requests,os; [os.system('git clone {[http_url_to_repo]}'.format(p)) for p in requests.get('https://<<REPO_URL>>/api/v4/groups/<<GROUP_ID>>',headers={'PRIVATE-TOKEN':'<<YOUR_PRIVATE_TOKEN>>'},verify=False).json()['projects']]
Replace <<REPO_URL>>, <<GROUP_ID>> and <<YOUR_PRIVATE_TOKEN>>
If you have far more then 20 projects in the group you have to deal with pagination https://docs.gitlab.com/ee/api/#pagination. That is you have to do several requests to clone all projects. The most challenging part is to obtain a full projects list in the group. This is how to do it in bash
shell using curl
and jq
:
for ((i = 1; i <= <NUMBER_OF_ITERATIONS>; i++)); do curl -s --header "PRIVATE-TOKEN: <YOUR_TOKEN>" "https://<YOUR_URL>/api/v4/groups/<YOUR_GROUP_ID>/projects?per_page=100&page=${i}" | jq -r ".[].ssh_url_to_repo"; done | tee log
The number of iterations in a for
loop can be obtained from X-Total-Pages
response header in a separate request, see Pagination in Gitlab API only returns 100 per_page max.
After projects list is obtained, you can clone projects with:
for i in `cat log`; do git clone $i; done

- 33,961
- 14
- 109
- 164
No need to write code, just be smart
In chrome we have extension that grabs all URL's "Link Klipper"
Extract all the URL's in to an excel file just filter them in excel
create
.sh
file#!/bin/bash git clone https:(url) (command)
run this script, done all the repository's will be cloned to your local machine at once

- 2,852
- 3
- 30
- 47
I was a little unhappy with this pulling in archived
and empty
repo's, which I accept are not problems everyone has. So I made the following monstrosity from the accepted answer.
for repo in $(curl -s --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.spokedev.xyz/api/v4/groups/238?include_subgroups=true" | jq '.projects[] | select(.archived == false) | select(.empty_repo == false) | .http_url_to_repo' | sed s"-https://gitlab-https://oauth2:\${GITLAB_TOKEN}@gitlab-g"); do
echo "git clone $repo"
done | bash
This is derived from the top answer to give full credit; but it just demonstrates selecting based on properties (for some reason === does not work). I also have this using HTTPS clone, because we set passwords on our SSH keys and don't want them in key-chain, or to type the password many times.

- 1,488
- 19
- 27
I was wrestling with issues with the scripts posted here, so I made a dumbed down hybrid version using an API call with PostMan, a JSON Query, and a dumb bash script. Here's the steps in case someone else runs into this.
- Get you group id. My group id didn't match what was displayed on my group's page for some reason. Grab it by hitting the api here: https://gitlab.com/api/v4/groups
- Use Postman to hit the API to list your projects. URL: https://gitlab.com/api/v4/groups/PROJECT_ID/projects?per_page=100&include_subgroups=true Set the Auth to use API Key, where the Key is "PRIVATE-TOKEN" and the value is your private API Key
- Copy the results and drop them in at: https://www.jsonquerytool.com/ Switch the Transform to JSONata. Change the query to "$.http_url_to_repo" (ssh_url_to_repo if using SSH)
- You should now have a JSON array of your git urls to clone. Change the format to match Bash array notation (change [] to () and drop the commas).
- Drop your Bash array into the repos variable of the script below.
repos=(
""
)
for repo in ${repos[@]}; do git clone $repo
done
- Save your script in the folder where you want your repos checked out.
- Run
bash {yourscriptname}.sh
- That should do it. You should now have a directory with all of your repos checked out for backup purposes.

- 1,087
- 1
- 13
- 28
It's my understanding that all the answers only allow you to clone repos, but not issues, boards, other settings, etc. Please correct me if I am wrong.
I feel if I want to back up all the data from multiple projects, the intention is to include not only repos, but also other data, which could be as important as the repos.
Self-hosted Gitlab instances can achieve this with official support, see backup and restore GitLab.

- 1,776
- 19
- 25