Automatically git clone into a unique destination directory

Question

I have a few hundred repos in a text file that I need to clone. A fair number of the repos have the same name (e.g., forks) and hence, the same destination directory. Is there a way to clone the repos with unique destination names?

Example:

for i in $(cat repolist.txt); do git clone $i $ORGNAME.$REPONAME; done

Or something like that?

What is the format of the `repolist.txt` file? Like an example url — smac89, Aug 20 '20 at 21:54
Just a list of repo URLs...one URL per line (e.g., https://github.com/rapid7/metasploit-framework) — Kinematics, Aug 20 '20 at 21:57
You can use `||` to combine commands such that the second one only executes if the first fails, and you can use `$(basename $(dirname $i))` to get the ORGNAME and `$(basename $i)` to get the REPONAME. — jeremysprofile, Aug 20 '20 at 22:16
You should read your file [differently](https://stackoverflow.com/q/10929453/3266847). — Benjamin W., Aug 21 '20 at 02:33

ErikMD · Accepted Answer · 2020-08-23T15:30:15.220

Assuming all your URLs have one of these forms:

https://github.com/user1/repo1.git
https://github.com/user2/repo1.git  
git@github.com:user3/repo2.git  
git@github.com:user4/repo2.git

you can define a Bash function like this:

clone() {
  local url=$1
  local repo=${url%.git} # remove .git suffix if need be
  repo=${repo##*/}       # remove largest prefix upto last slash
  local org=${url%/*}    # remove suffix from last slash
  org=${org##*[/:]}      # remove largest prefix upto slash or colon
  ( set -x;  # verbose mode
    git clone "$url" "$org.$repo" )
}

Then you can do indeed:

for i in $(cat repolist.txt); do clone "$i"; done

One advantage of using this syntax (${parameter#word}, ${parameter%word}, etc.) over basename and dirname is that no extra process is spinned, since this parameter expansion feature is builtin in Bash.

Finally, as explained in the SO question Read a file line by line assigning the value to a variable (mentioned by Benjamin W.), you can avoid the $(cat repolist.txt) phrasing and run the following command:

while IFS= read -r line || [ -n "$line" ]; do clone "$line"; done < repolist.txt

which is more efficient; and also more robust in case a line contains spaces (even if this situation won't occur here).

Automatically git clone into a unique destination directory

1 Answers1