0

I'm trying to do something really easy in order to learn how to use subprocess in python

What I'm trying is this:

ll | egrep "*gz"

so after read the manual of python (which I didn't understand very well), I tried this:

lista = subprocess.Popen(['ls', '-alF'], stdout=subprocess.PIPE)
filtro = subprocess.Popen(['egrep', '"*gz"'], stdin=lista.stdout, stdout=subprocess.PIPE)
filtro.communicate()[0]

But all I get is '' and I don't really know how to do this, I've read this but it seems I didn't get it at all... could somebody explain to me how this works in order to use it after with other commands??

Thanks in advance!!

Community
  • 1
  • 1
user1924264
  • 15
  • 5
  • Is the star supposed to be expanded to match glob expressions here? – mgilson Jan 30 '13 at 20:18
  • 1
    You put `lista` instead of `list`. Also not a good idea to use list as a name. – f p Jan 30 '13 at 20:18
  • @fp oh sorry, but that is not the error, I just tried to put it in english!! – user1924264 Jan 30 '13 at 20:19
  • Put `print filter.communicate()` to see if you get an error during execution. `list`is a name used in python so it's confusing if you use it as a variable name. – f p Jan 30 '13 at 20:24
  • Could you post exactly the code you are getting the wrong result with, not a translated version? I.e. as a self-contained script we can test? You seem to have understood how subprocess works, so the error is probably just a small bug somewhere. – amaurea Jan 30 '13 at 20:26
  • @fp `('', None)` this is the reulst of `print` @amaurea UPDATED!! – user1924264 Jan 30 '13 at 20:43

2 Answers2

1

The problem might be the double set of quotes around the argument to egrep. Try this instead:

import subprocess
ls = subprocess.Popen(['ls', '-alF'], stdout=subprocess.PIPE)
egrep = subprocess.Popen(['egrep', '\.gz$'], stdin=ls.stdout, stdout=subprocess.PIPE)
print egrep.communicate()[0]

I am assuming you are looking for files ending in ".gz" here, as your initial regex does not make sense. If you are simply looking for files ending in "gz", you would use 'gz$' instead. And if you do not case where in the line "gz" appears, simply use 'gz'.

Edit: Here is a full example. In a directory containing the three files "pipe.py", "test1.py.gz" and "test2.py.gz", where "pipe.py" is the above script, I execute:

$ python pipe.py

With the result

-rw-r--r-- 1 amaurea amaurea  146 Jan 30 20:54 test1.py.gz
-rw-r--r-- 1 amaurea amaurea  150 Jan 30 20:54 test2.py.gz
amaurea
  • 4,950
  • 26
  • 35
  • @user1924264 What files are in the directory you are running this in? Are there any .gz-files there? – amaurea Jan 30 '13 at 20:46
  • @user1924264 My test (which works for me) was in an interactive ipython shell. If you are running it as a normal python script, you need a print to actually see the output from communicate. I have added that to the code above now. – amaurea Jan 30 '13 at 20:52
  • @user1924264 The example above *works for me*. I will update the answer with a full example, how I run it, and what I get as output. – amaurea Jan 30 '13 at 20:53
  • wel... this is weird, I had to go and now when I come back, I did it again and it worked... what the heck?? – user1924264 Jan 31 '13 at 03:20
0

On Unix, take advantage of shell argument:

lista = subprocess.Popen('ls -alF', stdout=subprocess.PIPE, shell=True)
filtro = subprocess.Popen('egrep "*gz"', stdin=lista.stdout, stdout=subprocess.PIPE, shell=True)
filtro.communicate()[0]

You can simply copy commands and do not wary about breaking them into argument lists.

Also you can specify shell explicitly:

subprocess.Popen('bash -c "ls -l"', stdout=subprocess.PIPE, shell=True)
Yevgen Yampolskiy
  • 7,022
  • 3
  • 26
  • 23
  • yes, I did that and it worked, but it also says it is very insecure and if something had special characters is a pain in the @$$ to escape them, so I tried with the more secure way, if you read the last comment I did in the selected answer you'll see I already did this, thanks anyway!! – user1924264 Jan 31 '13 at 03:24
  • Sorry, did not read the whole thread. I found shell argument very handy in my projects, so I decided to share it – Yevgen Yampolskiy Jan 31 '13 at 03:28
  • Yes, thanks for that, and if I could give you an advice, you should try to do it in the way I did, cause is more secure, thanks for sharing!! – user1924264 Jan 31 '13 at 03:57