Unable to read file with python

Question

I'm trying to read the content of a file with python 3.8.5 but the output is empty, I don't understand what I'm doing wrong.

Here is the code:

import subprocess
import os

filename = "ls.out"
ls_command = "ls -la"
file = open(filename, "w")
subprocess.Popen(ls_command, stdout=file, shell=True)
file.close()

# So far, all is ok. The file "ls.out" is correctly created and filled with the output of "ls -la" command"

file = open(filename, "r")
for line in file:
    print(line)
file.close()

The output of this script is empty, it doesn't print anything. I'm not able to see the content of ls.out.

What is not correct here ?

Just curious - any reason not to use python to read directory content? — Patrick Artner, Mar 07 '21 at 11:34
Yes, "ls -la" was just an example. The exact command is more complex. I just didn't want to pollute the post with it. — iAmoric, Mar 07 '21 at 11:35
I tried communicating with the Popen and the output was also blank, so there might be some problem with how you ran the command. — Ismail Hafeez, Mar 07 '21 at 11:38
I just tried adding `p.terminate()` before closing the file and still blank output — iAmoric, Mar 07 '21 at 11:52
Does it work using the `with open(blarg) as file` context syntax? — xdhmoore, Mar 07 '21 at 11:56

score 7 · Accepted Answer · answered Mar 07 '21 at 11:57

Popen creates a new process and launches it but returns immediately. So the end result is that you've forked your code and have both processes running at once. Your python code in executing faster than the start and finish of ls. Thus, you need to wait for the process to finish by adding a call to wait():

import subprocess
import os

filename = "ls.out"
ls_command = "ls -la"
file = open(filename, "w")
proc = subprocess.Popen(ls_command, stdout=file, shell=True)
proc.wait()
file.close()


file = open(filename, "r")
for line in file:
    print(line)
file.close()

Thanks! This is what I was missing. Now it's working fine – iAmoric Mar 07 '21 at 12:00 — iAmoric, Mar 07 '21 at 12:00

score 1 · Answer 2 · answered Mar 07 '21 at 11:58

Popen merely starts the subprocess. Chances are the file is not yet populated when you open it.

If you want to wait for the Popen object to finish, you have to call its wait method, etc; but a much better and simpler solution is to use subprocess.check_call() or one of the other higher-level wrappers.

If the command prints to standard output, why don't you read it drectly?

import subprocess
import shlex

result = subprocess.run(
    shlex.split(ls_command), # avoid shell=True
    check=True, text=True, capture_output=True)
line = result.stdout

For (much) more on this topic, see https://stackoverflow.com/questions/4256107/running-bash-commands-in-python/51950538#51950538 — tripleee, Mar 07 '21 at 12:00

Unable to read file with python

2 Answers2