15

I once asked a very similar question and got a response that worked from the command line but am now wanting to use R to automate the process from Windows (Linux is much easier).

Here is what I'm trying to do:

  1. Create a local directory (or it exists already)
  2. Generate a new github repo in the cloud by the same name as local (based on this answer)
  3. Add a .git to the local repo
  4. Make an initial commit
  5. Establish link between cloud repo and local repo
  6. Push the commit and the files in the local repo to github

I believe based on the output that I get all the way to step 5 before I fail (as the commit and files from the local directory never go to github in the cloud). I know step 2 works because the empty repo is created here. I don't know how to test step 5. At the last step shell(cmd6, intern = T) RGui and RStudio result in an eternal death spiral. The question is: How can I push the commit and local repo to the cloud.

Here is my updated code (the only thing that is user specific is username and password in third code chunk):

## Create Directory
repo <- "foo5"
dir.create(repo)
project.dir <- file.path(getwd(), repo) 

## Throw a READ.ME in the directory
cat("This is a test", file=file.path(project.dir, "READ.ME"))

## Github info (this will change per user)
password <-"pass" 
github.user <- "trinker"  

## Get git location
test <- c(file.exists("C:/Program Files (x86)/Git/bin/git.exe"),
    file.exists("C:/Program Files/Git/bin/git.exe"))
gitpath <- c("C:/Program Files (x86)/Git/bin/git.exe",
  "C:/Program Files/Git/bin/git.exe")[test][1]

## download curl and set up github api
wincurl <- "http://curl.askapache.com/download/curl-7.32.0-win64-ssl-sspi.zip"
url <- wincurl
tmp <- tempfile( fileext = ".zip" )
download.file(url,tmp)
unzip(tmp, exdir = tempdir())       
shell(paste0(tempdir(), "/curl http://curl.haxx.se/ca/cacert.pem -o " , 
    tempdir() , "/curl-ca-bundle.crt"))
json <- paste0(" { \"name\":\"" , repo , "\" } ") #string we desire formatting
json <- shQuote(json , type = "cmd" )
cmd1 <- paste0( tempdir() ,"/curl -i -u \"" , github.user , ":" , password , 
    "\" https://api.github.com/user/repos -d " , json )

shell(cmd1, intern = T)

## Change working directory
wd <- getwd()
setwd(project.dir)

## set up the .git directory
cmd2 <- paste0(shQuote(gitpath), " init")
shell(cmd2, intern = T)

## add all the contents of the directory for tracking
cmd3 <- paste0(shQuote(gitpath), " add .")  
shell(cmd3, intern = T)       

cmdStat <- paste0(shQuote(gitpath), " status")  
shell(cmdStat, intern = T)

## Set email (may not be needed)
Trim <- function (x) gsub("^\\s+|\\s+$", "", x) #remove trailing/leading white 

x <- file.path(path.expand("~"), ".gitconfig")
if (file.exists(x)) {
    y <- readLines(x)
    email <- Trim(unlist(strsplit(y[grepl("email = ", y)], "email ="))[2])
} else {
    z <- file.path(Sys.getenv("HOME"), ".gitconfig")
    if (file.exists(z)) {
        email <- Trim(unlist(strsplit(y[grepl("email = ", y)], "email ="))[2])
    } else {
        warning(paste("Set `email` in", x))
    }
}
cmdEM <- paste0(shQuote(gitpath), sprintf(" config --global user.email %s", email))        
system(cmdEM, intern = T)

## Initial commit
cmd4 <- paste0(shQuote(gitpath), ' commit -m "Initial commit"')  
system(cmd4, intern = T) 

## establish connection between local and remote
cmd5 <- paste0(shQuote(gitpath), " remote add origin https://github.com/",
    github.user, "/", repo, ".git")  
shell(cmd5, intern = T) 

## push local to remote 
cmd6 <- paste0(shQuote(gitpath), " push -u origin master")  
shell(cmd6, intern = T) 

setwd(wd)

I know the script is a bit longer but it's all necessary to recreate the problem and replicate the issue:

Note I updated the question in light of Simon's response as he was correct and got closer to the push. The content of the original question can be found here.

Community
  • 1
  • 1
Tyler Rinker
  • 108,132
  • 65
  • 322
  • 519
  • Does the repo appear ok? Could be an issue with Windows locking files and not letting go. Also perhaps try changing `cmd3` to `paste0(shQuote(gitpath), " add -all")` and then also running `paste0(shQuote(gitpath), " status")` immediately afterwards. **AND** you might want to put a switch to determine the platform and if on Windows use `shell` not `system`. I find `shell` to be a bit more friendly on `Windows`. – Simon O'Hanlon Sep 06 '13 at 10:48
  • In my experience, you're almost always better off using `system` and not `shell`. – hadley Sep 06 '13 at 21:35

2 Answers2

8

If are using https address, then make sure that:

  • the environment variable %HOME% is defined
  • a _netrc file exists in it with the right credential to push back to your repo

That file shoud contains:

machine github.com
login username
password xxxx
protocol https

That works even if you have activated the recent two-factor authentication on GitHub.

Then your push won't time out:

cmd6 <- paste0(shQuote(gitpath), " push -u origin master")  
shell(cmd6, intern = T) 

This is easier than setting public/private ssh keys.


As the OP Tyler Rinker commented, setting %HOME% is illustrated in my other answer "Git - How to use .netrc file on windows to save user and password".
This is normally done by git-cmd.bat:

if not exist "%HOME%" @set HOME=%HOMEDRIVE%%HOMEPATH%
@if not exist "%HOME%" @set HOME=%USERPROFILE%

But you can do it manually as well.

Community
  • 1
  • 1
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • I'm not the most computer savvy. Could you expand a slight bit on what: `the environment variable %HOME% is defined` means? Does it mean: `Sys.getenv()["HOME"]`? – Tyler Rinker Sep 09 '13 at 23:09
  • I see this related post: http://stackoverflow.com/questions/6031214/git-how-to-use-netrc-file-on-windows-to-save-user-and-password – Tyler Rinker Sep 09 '13 at 23:29
  • @TylerRinker sorry for the late answer: your comments were right during my night time ;) – VonC Sep 10 '13 at 05:10
2

The problem simply seems to be mixing up ssh and https protocols.

Note the URLs should be:

#  https:
"https://github.com/<username>/<myrepo>.git"

#  ssh:
"git@github.com:<username>/<repo>.git"

You have:

cmd5 <- paste0(shQuote(gitpath), " remote add origin https://github.com:",
github.user, "/", repo, ".git") 
cat( cmd5 )
"... remote add origin https://github.com:trinker/foo2.git"

Just change cmd5 to

# Note the forward slash at EOL in place of the colon
cmd5 <- paste0(shQuote(gitpath), " remote add origin https://github.com/",
github.user, "/", repo, ".git")
"... remote add origin https://github.com/trinker/foo2.git"

It also couldn't hurt to run this immediately after git add .:

cmdStat <- paste0(shQuote(gitpath), " status")  
shell(cmdStat, intern = T)
Simon O'Hanlon
  • 58,647
  • 14
  • 142
  • 184
  • Simon you are correct. This gets me past the next hurdle. The last step is the `push -u origin master`. This results in a death spiral from RGui and from RStudio. I see you created a foo repo on github but it's gone. Did you get the push to work? If so any idea on what I'm not doing correctly? I have no error message since it's a death spiral. I updated the code/question above to show what I did. – Tyler Rinker Sep 06 '13 at 15:20
  • @TylerRinker hmm. I got a death spiral too. I am thinking `git` is just timing out. It's probably something pretty obvious, we're just missing it! I'll pop my thinking cap on after work (who thinks at work) :-) – Simon O'Hanlon Sep 06 '13 at 15:23
  • I am thinking that a `push` in RStudio requires my user name and password. That might mean that the API is needed here as well (this is just a thought I'm throwing out but don't yet fully understand how the API works). – Tyler Rinker Sep 07 '13 at 03:48
  • I started with `json2 <- " { \"events\":\"push\" } "` and repeated what you did for cmd1 but it failed as I didn't specify the repo but this formatting is very tricky. – Tyler Rinker Sep 07 '13 at 04:05
  • Yeah I think this is a problem trying to push over http. We should try to figure out using `ssh`... – Simon O'Hanlon Sep 07 '13 at 05:10