Python subprocess readlines()?

Question

So I'm trying to move away from os.popen to subprocess.popen as recommended by the user guide. The only trouble I'm having is I can't seem to find a way of making readlines() work.

So I used to be able to do

list = os.popen('ls -l').readlines()

But I can't do

list = subprocess.Popen(['ls','-l']).readlines()

would rather do `list = subprocess.call(['ls','-l']).readlines() ` — joedborg, Sep 19 '11 at 09:14
That's wrong -- `call` only returns the returncode, not the output. — agf, Sep 19 '11 at 09:18
http://stackoverflow.com/questions/2924310/whats-a-good-equivalent-to-pythons-subprocess-check-call-that-returns-the-conte — jfs, Sep 19 '11 at 09:53

Fred Foo · Answer 1 · 2011-09-19T09:32:20.840

39

ls = subprocess.Popen(['ls','-l'], stdout=subprocess.PIPE)
out = ls.stdout.readlines()

or, if you want to read line-by-line (maybe the other process is more intensive than ls):

for ln in ls.stdout:
    # whatever

edited Sep 19 '11 at 09:32

answered Sep 19 '11 at 09:17

Fred Foo

355,277
75
744
836

7

This approach is preferable to the accepted answer as it allows one to read through the output as the sub process produces it. – Rob Marrowstone Jul 21 '16 at 23:19
You need to iterate differently over `stdout`. – Mad Physicist Nov 18 '22 at 03:46
**Don't do that**, the Python3 documentation states the following: "Warning: Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process. " – ShellCode Jul 23 '23 at 17:28

score 32 · Accepted Answer · answered Sep 19 '11 at 09:17

32

With subprocess.Popen, use communicate to read and write data:

out, err = subprocess.Popen(['ls','-l'], stdout=subprocess.PIPE).communicate()

Then you can always split the string from the processes' stdout with splitlines().

out = out.splitlines()

answered Sep 19 '11 at 09:17

agf

171,228
44
289
238

Thought this was meant to be the "nice new way" of doing it. Thanks for the answer, I'll set it when the timeout reaches. I mihg tjust stick to popen. – joedborg Sep 19 '11 at 09:25
4

The problem with `communicate` is that you get all of the output at once, which in the case of a non-recursive `ls` is not likely a problem. By using the `stdout` member, you can read line-by-line (think `find`). – Fred Foo Sep 19 '11 at 09:33
@FredFoo That's right... I've created a little thread [here](https://stackoverflow.com/questions/60858329/custom-popen-communicate-method-gives-wrong-output) in order to try to address what you're mentioning in your comment... it's not working well yet though :) – BPL Mar 26 '20 at 00:43

score 13 · Answer 3 · edited May 23 '17 at 12:26

13

Making a system call that returns the stdout output as a string:

lines = subprocess.check_output(['ls', '-l']).splitlines()

edited May 23 '17 at 12:26

Community

1
1

answered Sep 19 '11 at 09:49

jfs

399,953
195
994
1,670

Nice, I guess it's not widely known because it's Python 2.7+ only. – agf Sep 19 '11 at 10:04
@agf: follow the link there is an adaption for older Python versions. – jfs Sep 19 '11 at 10:10
I know, I already followed both links you posted (and upvoted each post). I was just surprised that I didn't know about this convenience function until I saw it was 2.7+. – agf Sep 19 '11 at 10:15
This one is the recommended solution, if you are using 2.7+ – tbrittoborges Nov 19 '15 at 11:55

Kimvais · Answer 4 · 2011-09-19T11:46:22.033

3

list = subprocess.Popen(['ls', '-l'], stdout=subprocess.PIPE).communicate()[0].splitlines()

straight from the help(subprocess)

edited Sep 19 '11 at 11:46

answered Sep 19 '11 at 09:23

Kimvais

38,306
16
108
142

That's not a list of lines, it's a single multi-line string. – agf Sep 19 '11 at 09:26

score 3 · Answer 5 · edited Feb 09 '21 at 10:50

A more detailed way of using subprocess.

# Set the command
command = "ls -l"

# Setup the module object
proc = subprocess.Popen(command,
                    shell=True,   
                    stdin=subprocess.PIPE,
                    stdout=subprocess.PIPE,
                    stderr=subprocess.PIPE)

# Communicate the command   
stdout_value,stderr_value = proc.communicate()

# Once you have a valid response, split the return output    
if stdout_value:
    stdout_value = stdout_value.split()

Python subprocess readlines()?

5 Answers5

Linked