cURL doesn't work through subprocess in Python 3

Question

I can successfully get Docker with the following command in a bash script: curl -O https://download.docker.com/mac/stable/Docker.dmg > ./Docker.dmg

Now I want to execute the same in Python:

from subprocess import call
call(["curl", "-O",  "https://download.docker.com/mac/stable/Docker.dmg", ">", "./Docker.dmg"])

However, I am met with this error when running it:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  123M  100  123M    0     0  25.0M      0  0:00:04  0:00:04 --:--:-- 27.2M
curl: (6) Could not resolve host: >
curl: (7) Failed to connect to  port 80: Connection refused

It should be exactly the same bash script, but it doesn't work through the Python script. Should I add something more for it to work? Port?

Weird. I put the ">" inside with the ./"Docker.dmg" and it will work as intended, but still throw the error `curl: (6) Could not resolve host: >` ? Really strange. Python 3.6.2 — cbll, Sep 07 '17 at 10:18

Oleh Rybalchenko · Accepted Answer · 2017-09-07T10:37:05.253

Maybe this temporary solution helps you:

call(["curl -O https://download.docker.com/mac/stable/Docker.dmg > ./Docker.dmg"], shell=True)

Here is a good explanation how shell argument works.

UPDATE:

The thing is that for output redirection with > you need shell. To do this with subprocess, use stdout argument:

f = open('docker.dmg', 'w')
call(["curl", "-O",  "https://download.docker.com/mac/stable/Docker.dmg"], stdout=f)

score 1 · Answer 2 · answered Sep 07 '17 at 12:07

curl's man page has this to say about the -O option:

-O, --remote-name

Write output to a local file named like the remote file we get. (Only the file part of the remote file is used, the path is cut off.)

The file will be saved in the current working directory. If you want the file saved in a different directory, make sure you change the current working directory before invoking curl with this option.

The remote file name to use for saving is extracted from the given URL, nothing else, and if it already exists it will be overwritten. [...]

As such, it's in fact curl itself what is writing your file, not Python, and there is no need at all to further redirect and read stdout. Your code can thus be simplified to just this:

from subprocess import call
call(["curl", "-O",  "https://download.docker.com/mac/stable/Docker.dmg"])

This will download the Docker.dmg file and write it to the current directory. Note that it will overwrite an existing Docker.dmg file in the current directory if one exists already.

cURL doesn't work through subprocess in Python 3

2 Answers2