Clone multiple git repositories in one local directory?

Question

Is it possible to git clone multiple git repositories with one command (for example: git clone "1.git,2.git,3.git.." in one local directory?

What do you mean exactly? Do you mean to "merge' all the repos in a single one? — Atropo, Sep 19 '13 at 16:34
No. For example: I have: x.git, y.git, z.git ... To checkout them I have to do: git clone x.git, git clone y.git, git clone z.git, ... I want to git clone multiple git repos with 1 command. Is it possible? — vir2al, Sep 19 '13 at 16:39
Well, you could do it in one command *that you type*, but there will still be multiple commands that are actually executed. Something like `echo x.git y.git z.git | xargs -n 1 -d ' ' git clone`, for example. — twalberg, Sep 19 '13 at 17:54
If you use mu-repo, it has support for such a clone provided you commit a .mu_repo with dependency information (see: https://fabioz.github.io/mu-repo/cloning/) — Fabio Zadrozny, Aug 12 '16 at 12:38
Check out [tsrc](https://your-tools.github.io/tsrc/). https://stackoverflow.com/questions/816619/managing-many-git-repositories#comment130273168_816619 — Johann Chang, Sep 19 '22 at 15:49

score 25 · Answer 1 · answered May 14 '18 at 17:10

25

This is how almost solved for myself:

git clone https://myrepo.com/folder.git && \
git clone https://myrepo.com/folder2.git && \
git clone https://myrepo.com/folder3.git

This is way easier to build using a code editor like Sublime or VSCode.

The only downside for me: If you did not store your credentials, you're gonna have to type it over and over.

answered May 14 '18 at 17:10

Fábio Correia

595
7
16

This clones the repos sequentially but it was still useful. Thanks! – Luboš Turek May 28 '18 at 09:32
No problem! I am glad I heped some way :) – Fábio Correia May 31 '18 at 19:36
1

No need to repeat credentials if you put them in each URL: `https://username:password@myrepo.com/folder.git`. Probably less secure to include your password in plain text like that, but it suited my purposes well. – Oct 10 '18 at 19:21
I created the list of repos in a text file similar to how it's in this answer but without the `&& \ ` at the end of each line, pasted them in PowerShell, hit enter and entered my credentials just **once**, after that it cloned all the repos one by one. For eg: Mine looked like this: `git clone https://myrepo.com/folder1/ git clone https://myrepo.com/folder2/` – Ash K Dec 21 '22 at 17:59

VonC · Answer 2 · 2019-05-06T08:12:22.527

You can find script example like this one:

I have this file called "clone" containing URLs of several git repos (taken from djangosites.com. Awesome site. Must visit)

Snippet:

$ cat clone
https://github.com/igorsobreira/igorsobreira.com https://github.com/ella/ella https://github.com/divio/django-cms/ https://github.com/palewire/palewire.com https://github.com/jabapyth/jfcom https://github.com/humanfromearth/snippify https://github.com/scaphilo/koalixcrm https://github.com/jlev/Boycott-Toolkit https://github.com/jbalogh/zamboni/ https://github.com/ASKBOT/askbot-devel https://github.com/emesik/djiki https://github.com/vicalloy/LBForum https://github.com/agiliq/agiliq https://github.com/bartTC/dpaste.de https://github.com/bartTC/django-paste https://github.com/bartTC/dpaste_de/ https://github.com/fotochest/fotochest https://esp.mit.edu/git/esp-project.git https://github.com/titan2x/bashoneliners.git

Apparently, it's harder to clone multiple repos at once (git clone <repo1> <repo2> ... <repon> does not work). So I wrote this short bash code to make it work:

Code:

atm in /home/atm/git/django_repos
$ for f in `cat clone`; do `git clone $f`; done

You would find many more on gist.github.com, like this one, to clone all your repos from GitHub:

#!/bin/bash
#
# Copyright 2011, Tim Branyen @tbranyen <tim@tabdeveloper.com>
# Dual licensed under the MIT and GPL licenses.
#
# Automatically clone single or multiple repos into a folder, 
# great for setting up a git projects folder.
#
# Install: curl https://gist.github.com/raw/902154/github.sh > /usr/local/bin/gh
#          chmod +x /usr/local/bin/gh
#

# Internal properties
GITHUB_PREFIX=git@github.com:
GITHUB_USERNAME=$(git config --global github.user)

function main {
  # Improperly configured user
  detect_user

  # Missing arguments
  args=$1
  if [ -z $args ]; then
    echo '
      gh: try ''`gh --help`'' for more information
    '
    exit
  fi

  # Display help text
  if [ $args = '--help' ]; then
    echo '
      Clone repos from your GitHub
        gh repo1 repo2

      Clone repos from others GitHub
        gh username/repo1 username/repo2

      Clone mixed repos:
        gh repo1 username/repo2

      Clone line separated repos from file:
        cat file | xargs gh
    '
    exit
  fi

  # Parse arguments and clone repos.
  find_repos
}

function detect_user {
  # If no username configured, attempt to pull from git --config
  if [ -n "$GITHUB_USERNAME" ]; then
    USERNAME=$GITHUB_USERNAME
  else
    echo '
      gh: missing username
      configure username with ''`git config --global github.user username`''
    '
    exit
  fi
}

function find_repos {
  for repo in $args; do
    # If a user provides the parameter username/repo pull in that specific repository.
    if [ `awk -v repo="$repo" -v delimit="/" 'BEGIN{print index(repo,delimit)}'` -ne 0 ]; then
      echo "Pulling in $repo";
      git clone $GITHUB_PREFIX$repo.git

    # Default to you.
    else
      echo "Pulling in $USERNAME/$repo";
      git clone $GITHUB_PREFIX$USERNAME/$repo.git
    fi
  done
}

main $*

More generally, a scripting approach is needed, and lilgeek mentioned bl4ckbo7/agt, a python script which includes cloning with fastest and parallel clone processing feature.

score 17 · Answer 3 · answered Dec 01 '19 at 21:08

If all repositories are hosted under the same namespace (username), you can do the following:

$ echo name1 name2 name3 | xargs -n1 | xargs -I{} git clone https://github.com/username/{}

Explanation

First part will split the names separated by space into multiple lines
```
$ echo name1 name2 name3 | xargs -n1
name1
name2
name3
```
Then, each of these names is passed separately to the next xargs call, which invokes git clone and replaces {} substring in URL with repository name, which basically translates to
```
$ git clone https://github.com/username/name1
$ git clone https://github.com/username/name2
$ git clone https://github.com/username/name3
```

In case not all repos are hosted under the same namespace, you can move dynamic parts to echo part, and keep common part of the URL in the last part.

Worked fine! to cache the credential : `git config --global credential.helper cache` `git config --global credential.helper 'cache --timeout=3600'` For details see [@chase-roberts answer](https://stackoverflow.com/a/28487383/7338187) — bareMetal, Aug 28 '20 at 10:58

score 6 · Answer 4 · answered Dec 13 '21 at 09:06

6

Clone repositores one by one into the same directory:

xargs -L1 git clone <<EOF
https://github.com/motdotla/dotenv.git
https://github.com/node-fetch/node-fetch.git
EOF

answered Dec 13 '21 at 09:06

Iceberg

2,744
19
19

1

Simple and clean. That should be the correct answer – Renato Rodrigues Oct 06 '22 at 14:23

score 4 · Answer 5 · answered Mar 25 '20 at 10:23

If you use windows, you can use powershell.

Just make list in favourite text editor like this:

git clone https://github.com/1.git;
git clone https://github.com/2.git;
git clone https://github.com/3.git;
git clone https://github.com/4.git;
git clone https://github.com/5.git;

In powershell console go do your directory.

Example

cd d:/myGitRepos

Then copy repo list directly do powershell console. First time it will ask username and password, but after that it will remember it.

score 2 · Answer 6 · answered Nov 03 '17 at 12:52

I have created a sample script for my project. Our project includes lots of repositories which needs to be cloned when a fresh person joins the team.

So here is the content of script, which you can save into some executable files and execute.

   echo -n Please Enter your GitHub Username:
        read username
        echo -n Please Enter your GitHub Password
        read -s password // doesn't echoes the password on screen
        git clone 

https://$username:$password@github.com/location/reponame1.git
https://$username:$password@github.com/location/reponame2.git
....
https://$username:$password@github.com/location/reponamen.git

Its quiet useful to avoid boring manual efforts and ensure that all required projects are cloned in one go.

Hope my answer helps someone else as well.

Yeah.. I figured out..there is a problem with the script.. if your password contains @ it doesnt work as the command itself as @ after password. Will have to work on it. But it works otherwise. — Sanjay Bharwani, Mar 05 '18 at 10:36

Renato Back · Answer 7 · 2021-04-16T13:15:26.210

2

You can use a mix of solutions.

Use git's credential.helper to set a cache timeout (see this), then you can use a script/list like the one suggested by Fábio. This way, you'll only have to type your credentials once (usually, unless a clone takes longer than the cache timeout.

git config --global credential.helper 'cache timeout 3600'
# In some clients, the above line might only work with double quotes
# git config --global credential.helper "cache timeout 3600"
git clone https://myuser@bitbucket.org/myproject/myrepo.git
### password should be prompted 
git clone https://myuser@bitbucket.org/myproject/mysecondrepo.git
### look ma, no password prompt!
git clone https://myuser@bitbucket.org/myproject/mythirdrepo.git
git clone https://myuser@bitbucket.org/myproject/myfourthrepo.git
git clone https://myuser@bitbucket.org/myproject/andsoforthrepo.git

That's still sequential, but it helps and is pretty simple, IMHO.

edited Apr 16 '21 at 13:15

answered Jul 24 '18 at 02:09

Renato Back

706
1
10
21

Kindly correct the timeout syntax as ```git config --global credential.helper "cache --timeout=3600" ```. Then only your answer worked for me. – Kirubakaran Apr 04 '21 at 17:36
Probably has something to do with the terminal client you're using, @Kirubakaran I'll add a note to it. Thanks pointing it out. – Renato Back Apr 16 '21 at 13:11
I was using the gnome-terminal and bash shell. – Kirubakaran Apr 22 '21 at 03:28

score 1 · Answer 8 · answered Sep 03 '20 at 22:30

i had the same issue , and i wrote this simple bash to do it

#!/bin/bash


#get repo number
read -p "how many repo you want to clone:" reponum


#take username&password

read -p "enter git username: " username 
read -p "enter git password: " password

#urlencode password to use it in https without any issues

epassword=$(php -r "echo urlencode('$password');")

while ((reponum--))
do
    #take repo name without .git at the end
    read -p "enter repo name: " reponame

    git clone "https://$username:$epassword@github.com/$username/$reponame.git"
done

score 1 · Answer 9 · answered Apr 06 '21 at 07:55

Hi assuming you want to do this over ssh not https, and you have your ssh key setup. Also you might have multiple repositories in the same org you could use a for in bash.

BASE='git clone git@github.com:/myrepo.com/'
while IFS= read -r THELINE; do
 $BASE"$THELINE"
done </PATHTOLISTFILE

Make sure in the list you have the path to a file with all the repos, essentially you are getting:

git clone git@github.com:/myrepo.com/folder1

git clone git@github.com:/myrepo.com/folder2

score 0 · Answer 10 · answered Aug 23 '19 at 02:10

After reading this and other posts I ended with this script.

I needed authentication on my local environment for my repositories.

So,the script is going to ask you the user, password and an option in case you want to remember your credentials on the repository.

For security reason I am erasing the user and password from the config file in git. (Not sure if those credentials are stored in another place)

#!/bin/bash
#
#This script is intended to be used with repositories that need authentication
#

#Funtion to erase the $username:$password@ from the config file from git in the repo
function erase_user_pass() {
  printf "${green}lets clean $1 from $2 \n${normal}"
  sed -i 's/'"$1"'/'""'/g' $2
}

#function to download the repo
#1-clone the repo
#2-fix the config file by deleting the user and password
#3-execute the git command to automatically store the credentials if the user answer yes on the startup
#
# param 1 = name of the repo
# param 2 = string to complet the url of the repo
#
# example repo in https://git.something.com/yourcompany/your.project.git
#
# the param1 should be your.project
# the param2 should be git.something.com/yourcompany/your.project.git (without 'https://')
#
# download_repo 'your.project' 'git.something.com/yourcompany/your.project.git'
#
function download_repo() {
  path=$(pwd)
  printf "${blue}\n Importing $1\n\n${normal}"
  git clone https://$username:$password@$2
  file="$(pwd)/$1/.git/config"
  replace_string="$username:$password@"
  erase_user_pass $replace_string $file
  cd ./$1
  if [ "$STORE_OK" == 'Yes' ]
    then
     printf "${green} store credentials for the repo $1  \n${normal}"
     git config credential.helper store
  fi
  cd $path
}

blue=$(tput setaf 4)
green=$(tput setaf 2)
normal=$(tput sgr0)

echo -n Please Enter your GitHub Username:
read username
echo -n Please Enter your GitHub Password:
read -s password

printf "\n\nDo you want to store your credentials locally?:"
STORE_OK=
select STORE_OK in Yes No
do
  case $STORE_OK in
    '') echo 'Please enter 1 for Yes, or 2 for No.';;
    ?*) break
  esac
done

printf "${blue}\n\n lets import your repos\n\n${normal}"

# repo 1 https://git.something.com/yourcompany/your.project.git
download_repo 'your.project' 'git.something.com/yourcompany/your.project.git'
# repo 2 https://git.something.com/yourcompany/your.project2.git
download_repo 'your.project2' 'git.something.com/yourcompany/your.project2.git'

printf "${blue}\n Enjoy :)"

exit 0

You should need to replace the calls to download_repo to point to your repositories, these are fake.

Regards and thanks to others answers.

score 0 · Answer 11 · answered Nov 02 '20 at 21:23

0

If you are cloning many repositories from one directory this is a good solution

eval $(printf "git clone \"%s\" & " <git_repo_directory>/*)

where <git_repo_directory> must be replaced with the path containing the git repositories

answered Nov 02 '20 at 21:23

J. Rehbein

107
1
9

score 0 · Answer 12 · answered Dec 29 '20 at 12:32

And here is a simplistic cmd version:

@echo off
SET repos=1.git,2.git,3.git,4.git
SET baseUrl=https://BaseUrl.Company.Com/Project/Whatever/

FOR %%r IN (%repos%) DO (
    git clone %baseUrl%%%r
)

Optionally, you can also set username/password in your bat file OR ask for user input with SET /p username="Username : " etc.

kkpareek · Answer 13 · 2022-02-04T06:07:12.633

0

Multiple repo clone via SSH:

NOTE: Only an authorized user will be able to use this script.

#!/bin/bash

#get repo count
read -p "how many repo you want to clone:" repo_count

while [ $repo_count -gt 0 ]
do
    #take repo name without .git at the end
    read -p "enter repo name: " reponame

    git clone "Enter your base URL here/$reponame.git"
    repo_count=$(( $repo_count - 1 ))
done

edited Feb 04 '22 at 06:07

answered Feb 03 '22 at 09:49

kkpareek

450
1
5
15

score 0 · Answer 14 · answered Feb 04 '22 at 13:17

I found this PowerShell script that worked for me:

# import multiple remote git repositories to local Source dir

 param (
  [string]$localFolder = "C:\repos\",
  [array]$repos = @("repo1", "repo2")
 )
 
# change the repoLocation based on your github access 
$repoLocation = "https://github.com/" 
$gitExtension = ".git"

# for each repo found remotely, check if it exists locally
# if dir exists, skip, if not, clone the remote git repo into it
foreach ($gitRepo in $repos) {
    If (Test-Path $localFolder\$gitRepo) {
        echo "repo $gitRepo already exists"
    }
    Else {
        echo "git clone $repoLocation$gitRepo$gitExtension $localFolder\$gitRepo"
        git clone $repoLocation$gitRepo$gitExtension $localFolder\$gitRepo
    }
}

score 0 · Answer 15 · answered Jul 05 '23 at 10:35

Interesting variety of answers, some which, personally seem unnecessarily complicated.

The following implementation worked for me brilliantly, (KISS approach):

Assuming the following:

github link: https://github.com/building-microservices-with-go/chapter
download chapters in range: 1-9 command:

for i in {1..9}; do git clone https://github.com/building-microservices-with-go/chapter$i; done

Of course, if the repos are not ordered simply as above example, this would be better to be read from the file then, for example as in @ChrisJ example.

Clone multiple git repositories in one local directory?

15 Answers15

Linked

Related