Shell script to clone a GitHub Repo

Question

I am trying to automate a process that contains a series of git commands.

I want the shell script to deal with some interactive commands, like passing the username and password to git clone url -v. I verified that if I just run git clone url -v it will show the following in order:

cloning into someRepo
asking for username
asking for password

I've tried:

echo -e 'username\n' | git clone url -v
echo -e 'username\npassword\n' | git clone url -v
git clone url -v <<< username\npassword\n
(sleep 5;echo -e 'username\n' | git clone url -v)

I thought that the first message cloning into repo will take some time. None of them is working, but all of them are showing the same message that Username for url:

Having spent lots of time in this, I know that

git clone https://$username:$password@enterpriseGithub.com/org/repo

is working, but it is UNSAFE to use since the log show the username and password explicitly.

Well-written software doesn't prompt for passwords on stdin, but goes straight to the TTY for security reasons. If you want to pass a password out-of-band, stdin is the wrong way to do it -- git has its own mechanisms to let you store config files in the user's dotfiles. Even better is to use SSH-agent-based authentication, potentially with the agent proxying to RSA authentication done on a physical authentication token (a YubiKey, smartcard, etc). — Charles Duffy, Dec 07 '18 at 16:58
Anyhow, re: doing things The Right Way in git, https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage is probably a reasonable place to start. — Charles Duffy, Dec 07 '18 at 17:00
...doesn't help you if the prompt isn't coming from stdin, but as an aside, it's better to use `printf '%s\n' "first line" "second line" "third line"` -- that way format sequences inside the individual variables aren't expanded, and you aren't depending on any behavior that isn't given in the POSIX spec (see http://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html, especially the APPLICATION USAGE and RATIONALE sections; `echo` is somewhat stunningly mutually incompatible even across standard-compliant implementations). — Charles Duffy, Dec 07 '18 at 17:02
Is possible that the first-time user can create and store config files via command lines only(and NOT manually interacting with the prompt) ? — , Dec 07 '18 at 17:03
@Charles Duffy, ah sorry, I meant using command line only and at the same time not manually reacting to the interactive prompt. Like before the job starts, user could specify the username and password parameters, and once the job starts, the process will solely depend on the script and not human interaction any more. — , Dec 07 '18 at 17:06
Right -- set up a credential helper (as described in the link in my 2nd comment), and there you are. — Charles Duffy, Dec 07 '18 at 17:09
As an aside, an obvious problem with your attempt is that variables are not expanded inside single quotes. This is a common FAQ; https://stackoverflow.com/questions/6697753/difference-between-single-and-double-quotes-in-bash — tripleee, Dec 07 '18 at 17:38

Charles Duffy · Answer 1 · 2018-12-07T17:29:26.137

Better practice would be to avoid user/password authentication at all (as by configuring agent-based auth, ideally backed by private keys stored on physical tokens), or set up credential storage in a keystore provided (and hopefully secured) by your operating system -- but if you just want to keep credentials off the command line, that can be done:

# Assume that we already know the credentials we want to store...
gitUsername="some"; gitPassword="values"

# Create a file containing the credentials readable only to the current user
mkdir -p "$HOME/.git-creds/https"
chmod 700 "$HOME/.git-creds"
cat >"$HOME/.git-creds/https/enterprise-github.com" <<EOF
username=$gitUsername
password=$gitPassword
EOF

# Generate a script that can retrieve stored credentials
mkdir -p -- "$HOME/bin"
cat >"$HOME/bin/git-retrieve-creds" <<'EOF'
#!/usr/bin/env bash
declare -A args=( )
while IFS= read -r line; do
  case $line in
    *..*) echo "ERROR: Invalid request" >&2; exit 1;;
    *=*)  args[${line%%=*}]=${line#*=} ;;
    '')   break ;;
  esac
done

[[ ${args[protocol]} && ${args[host]} ]] || {
  echo "Did not retrieve protocol and host" >&2; exit 1;
}
f="$HOME/.git-creds/${args[protocol]}/${args[host]}"
[[ -s $f ]] && cat -- "$f"
EOF
chmod +x "$HOME/bin/git-retrieve-creds"

# And configure git to use that 
git config --global credential.helper "$HOME/bin/git-retrieve-creds"

Shell script to clone a GitHub Repo

1 Answers1