Syntax for subprocess.call (Win7 x64)

Question

I am trying to call an .exe file that's not in my local Python directory using subprocess.call(). The command (as I type it into cmd.exe) is exactly as follows: "C:\Program Files\R\R-2.15.2\bin\Rscript.exe" --vanilla C:\python\buyback_parse_guide.r

The script runs, does what I need to do, and I have confirmed the output is correct.

Here's my python code, which I thought would do the exact same thing:

## Set Rcmd
Rcmd = r'"C:\Program Files\R\R-2.15.2\bin\Rscript.exe"'
## Set Rargs
Rargs = r'--vanilla C:\python\buyback_parse_guide.r'

retval = subprocess.call([Rcmd,Rargs],shell=True)

When I call retval in my Python console, it returns 1 and the .R script doesn't run, but I get no errors. I'm pretty sure this is a really simple syntax error... help? Much thanks!

Spaces in file paths can cause issues. Install R in a directory path without spaces. — mnel, Nov 30 '12 at 00:20
I don't see how this would be the cause, but I will try moving the installation to a different directory and report back. Thanks. — Ray, Nov 30 '12 at 00:22
I moved my R installation to a folder w/o spaces, re-tried, and still nothing. — Ray, Nov 30 '12 at 00:30
Does the command line work if you `cd` into the directory where your Pythons script is instead of the one where your R script is? — abarnert, Nov 30 '12 at 00:38
I `cd`-ed to `c:/python` where both scripts are. Then I re-ran the command from my question, and it runs. I don't see how `cd` in cmd.exe has anything to do with anything though. — Ray, Nov 30 '12 at 00:43
@RayBao: It could easily have been something with your script making assumptions about the currently being the same as the script directory. People do that all the time, especially on Windows. — abarnert, Nov 30 '12 at 00:45
Have you tried using `os.system` instead of `subprocess.call` as wim suggested? (It's not a good fix, but it is a good debugging step.) Can you put some logging into the R script to see why it's returning early? Or run it with `--verbose`? — abarnert, Nov 30 '12 at 00:46
Hold on… is Rscript even supposed to work on Windows? [The documentation](http://stat.ethz.ch/R-manual/R-patched/library/utils/html/Rscript.html) says "Rscript is only supported on systems with the execv system call", and it also says the only reason it's there is to handle #! scripts, which don't exist on Windows. Are you sure you don't want to just run R.exe? — abarnert, Nov 30 '12 at 00:51
I did, and it returns 1. There doesn't appear to be anything wrong with the .R script. I typed the following: `"C:\R\R-2.15.2\bin\i386\Rscript.exe" --vanilla --verbose C:\python\buyback_parse_guide.r` in `cmd.exe` and it did exactly what it should — Ray, Nov 30 '12 at 00:52
If `Rscript` didn't work on Windows, then why does calling the command from `cmd.exe` execute the .R script? — Ray, Nov 30 '12 at 00:53
Here's more information on R.exe, Rscript.exe: http://stackoverflow.com/questions/3412911/difference-between-r-exe-rcmd-exe-rscript-exe-and-rterm-exe Typing `os.system(Rcmd)` with `R.exe` opens an R command prompt. — Ray, Nov 30 '12 at 01:05
The point of passing `--verbose`, etc. was to pass it from the Python script, to see if it caused anything different to get printed. — abarnert, Nov 30 '12 at 01:06
Thank you for linking to the same answer I linked to in my answer. Did you actually read it? Of course `R.exe` opens an R prompt; that's what it does if you don't give it input. Among the many ways you could do that, since you're using the shell, you can just add `r'< C:\python\buyback_parse_guide.r'` to the `subprocess.call`/`os.popen` line. — abarnert, Nov 30 '12 at 01:08
I tried: `subprocess.call(r'"C:\R\R-2.15.2\bin\i386\R.exe" < C:\python\buyback_parse_guide.r',shell=True)` and it returns 2. I think I'll need to pass it the R `source(filepath)` command. Lemme try that... — Ray, Nov 30 '12 at 01:16
When I type `"C:\R\R-2.15.2\bin\i386\R.exe" CMD BATCH C:\python\buyback_parse_guide.r` into `cmd.exe`, the .R script runs successfully. What's the proper syntax for passing this into python? Really appreciate the help by the way. This is my 2nd day coding python. — Ray, Nov 30 '12 at 01:28

score 1 · Answer 1 · answered Nov 30 '12 at 00:32

To quote the docs:

If shell is True, it is recommended to pass args as a string rather than as a sequence.

Splitting it up (either manually, or via shlex) just so subprocess can recombine them so the shell can split them again is silly.

I'm not sure why you think you need shell=True here. (If you don't have a good reason, you generally don't want it…) But even without shell=True:

On Windows, if args is a sequence, it will be converted to a string in a manner described in Converting an argument sequence to a string on Windows. This is because the underlying CreateProcess() operates on strings.

So, just give the shell the command line:

Rcmd = r'"C:\Program Files\R\R-2.15.2\bin\Rscript.exe" --vanilla C:\python\buyback_parse_guide.r'
retval = subprocess.call(Rcmd, shell=True)

I tried your code-snippet above. Called `retval` and it still returns `1`. The .R script did not run. :? — Ray, Nov 30 '12 at 00:37

score 0 · Answer 2 · edited Jun 20 '20 at 09:12

According to the docs, Rscript:

… is an alternative front end for use in #! scripts and other scripting applications.

… is convenient for writing #! scripts… (The standard Windows command line has no concept of #! scripts, but Cygwin shells do.)

… is only supported on systems with the execv system call.

So, it is not the way to run R scripts from another program under Windows.

This answer says:

Rscript.exe is your friend for batch scripts… For everything else, there's R.exe

So, unless you have some good reason to be using Rscript outside of a batch script, you should switch to R.exe.

You may wonder why it works under cmd.exe, but not from Python. I don't know the answer to that, and I don't think it's worth digging through code or experimenting to find out, but I can make some guesses.

One possibility is that when you're running from the command line, that's a cmd.exe that controls a terminal, while when you're running from subprocess.call(shell=True) or os.system, that's a headless cmd.exe. Running a .bat/.cmd batch file gets you a non-headless cmd, but running cmd directly from another app does not. R has historically had all kinds of complexities dealing with the Windows terminal, which is why they used to have separate Rterm.exe and Rcmd.exe tools. Nowadays, those are both merged into R.exe, and it should work just fine either way. But if you try doing things the docs say not to do, that may not be tested, it's perfectly reasonable that it may not work.

At any rate, it doesn't really matter why it works in some situations even though it's not documented to. That certainly doesn't mean it should work in other situations it's not documented to work in, or that you should try to force it to do so. Just do the right thing and run R.exe instead of Rscript.exe.

Unless you have some information that contradicts everything I've found in the documentation and everywhere else I can find, I'm placing my money on Rscript.exe itself being the problem.

You'll have to read the documentation on the invocation differences between Rscript.exe and R.exe, but they're not identical. According to the intro docs,:

If you just want to run a file foo.R of R commands, the recommended way is to use R CMD BATCH foo.R

According to your comment above:

When I type "C:\R\R-2.15.2\bin\i386\R.exe" CMD BATCH C:\python\buyback_parse_guide.r into cmd.exe, the .R script runs successfully. What's the proper syntax for passing this into python?

That depends on the platform. On Windows, a list of arguments gets turned into a string, so you're better off just using a string so you don't have to debug the joining; on Unix, a string gets split into a list of arguments, so you're better off using a list so you don't have to debug the joining.

Since there are no spaces in the path, I'd take the quotes out.

So:

rcmd = r'C:\R\R-2.15.2\bin\i386\R.exe CMD BATCH C:\python\buyback_parse_guide.r'
retval = subprocess.call(rcmd)

Thanks for this. It looks like `Rscript.exe` is not the way to go. Still trying to get the syntax down using `R.exe` however.... — Ray, Nov 30 '12 at 01:30

Syntax for subprocess.call (Win7 x64)

2 Answers2