1

I am downloading a file using wget in Python using the code below:

p1 = subprocess.Popen(['wget',
                       '-P', 
                       'path/to/folder','http://www.ncbi.nlm.nih.gov/Traces/wgs/?download=ACHI01.1.fsa_nt.gz'],
                       stdout=subprocess.PIPE)

p1.studout.close()

The file gets downloaded and saved correctly in the given folder but the process keeps running. I tried p1.kills() but that doesn't work either. Please advise.

Thanks!

rafaelc
  • 57,686
  • 15
  • 58
  • 82
Prasad
  • 13
  • 2
  • have you tried `p1.terminate()`? – Samuel O'Malley Jun 28 '15 at 23:26
  • Because you use `stdout=subprocess.PIPE`, you have to read stdout by `p1.communicate()`. – Nizam Mohamed Jun 29 '15 at 00:01
  • As an aside there's a third party wget module https://pypi.python.org/pypi/wget or use urllib2 (see here http://stackoverflow.com/questions/24346872/python-equivalent-of-a-given-wget-command). Philosophically, I think, wrapping linux cmds is not really the pythonic way of doing things (usually the "best" way is to call the underlying c libs directly using a third party python module someone else wrote for you). (no idea if this is what the wget python module does though). – demented hedgehog Jun 29 '15 at 00:13
  • Also this http://urlgrabber.baseurl.org/ suggested in this post http://stackoverflow.com/questions/2467609/using-wget-via-python – demented hedgehog Jun 29 '15 at 00:15
  • have you tried `urllib.urlretrieve()` instead? – jfs Jun 29 '15 at 07:55

2 Answers2

1

Use subprocess.call

import subprocess 

subprocess.call (['wget', '-P', '/', 'http://www.ncbi.nlm.nih.gov/Traces/wgs/?download=ACHI01.1.fsa_nt.gz'])
Stefan Gruenwald
  • 2,582
  • 24
  • 30
  • Thanks everyone for the quick replies. using subprocess.call worked great !! – Prasad Jun 29 '15 at 01:23
  • @user1777629: don't use `stdout=PIPE` with `subprocess.call()` function; it may hang the child process if it generates enough output to fill the OS pipe buffer. – jfs Jun 29 '15 at 07:54
0

A call to wait() is missing after the Popen, to make the main process wait till the child process is terminated.. The following code seems to work fine:

import subprocess as sp

def main(args):
p1 = sp.Popen(['wget', '-P path',
               'http://www.ncbi.nlm.nih.gov/Traces/wgs/?download=ACHI01.1.fsa_nt.gz'],
               stdout = sp.PIPE)
p1.wait()
return 0

if __name__ == '__main__':
    import sys
    sys.exit(main(sys.argv))

(You can also group commandline parameters with their values, if they have any).

jcoppens
  • 5,306
  • 6
  • 27
  • 47
  • 1
    don't use `stdout=PIPE` unless you read from the pipe; it may block the subprocess. For example, try to run `python -c "print(chr(97)*10**2)"` as a subprocess. It should work as is with `10**2` characters. Change it to `10**5` or `10**6`, to see that it blocks forever. Replace `p1.wait()` with `output = p1.communicate()[0]`, to see that it does not block forever if the pipe is consumed. [To discard output, use `DEVNULL` instead](http://stackoverflow.com/a/11270665/4279) – jfs Jun 30 '15 at 10:56
  • Hi @J.F.Sebastian, thanks for your suggestion. I changed the code to not use stdout and I am just using subprocess.call. – Prasad Jun 30 '15 at 15:27