python subprocess output to list or file

Question

I want to run the following bash command in Python 3:

ls -l

I know that I can do the following:

from subprocess import call
call(['ls', '-l'])

How do I save this output to a file, or put it into lists or sets?

[-rw-r--r--]  [1] [name]  [staff]   [426] [14 Jan 21:52] [HelloWorld.class]
[-rw-r--r--@] [1] [name]  [staff]   [107] [14 Jan 21:51] [HelloWorld.java]
...
etc.

I want to be able to access particular information directly, and then add it to the set, but I do not know how many items will be listed.

Any hints, snippets, or examples would really help.

If you want to access file/dir information you might want to check the built-in `os` library: [documentation](http://docs.python.org/py3k/library/os.html). — Rik Poggi, Jan 16 '12 at 13:47

Gary van der Merwe · Answer 1 · 2021-03-09T14:17:54.373

With >= python3.5 you can use subprocess.run:

ls_lines = subprocess.run(['ls', '-l'], stdout=PIPE).stdout.splitlines()

With >= python2.7 or >= python3.0 you can use subprocess.check_output:

ls_lines = subprocess.check_output(['ls', '-l']).splitlines()

Prior to python2.7, you need to use the lower level api, which is a bit more involved.

ls_proc = subprocess.Popen(['ls', '-l'], stdout=subprocess.PIPE)
ls_proc.wait()
# check return code
ls_lines = ls_proc.stdout.readlines()

jcollado · Accepted Answer · 2012-01-16T13:44:16.847

14

One way to access to the information in ls -l output is to parse it. For example, csv.DictReader could be use to map each column to a field in a dictionary:

import subprocess
import csv

process = subprocess.Popen(['ls', '-l'], stdout=subprocess.PIPE)
stdout, stderr = process.communicate()

reader = csv.DictReader(stdout.decode('ascii').splitlines(),
                        delimiter=' ', skipinitialspace=True,
                        fieldnames=['permissions', 'links',
                                    'owner', 'group', 'size',
                                    'date', 'time', 'name'])

for row in reader:
    print(row)

The code above will print a dictionary for each line in ls -l output such as:

{'group': '<group_name>',
 'name': '<filename>',
 'links': '1',
 'date': '<modified_date>',
 'time': '<modified_time>',
 'owner': '<user_name>',
 'permissions': '-rw-rw-r--',
 'size': '<size>'}

edited Jan 16 '12 at 13:44

answered Jan 16 '12 at 13:37

jcollado

39,419
8
102
133

I get: ` File "ip4.py", line 16 print row ^ SyntaxError: invalid syntax` – beoliver Jan 16 '12 at 13:42
@user969617 Yes, sorry, that was for python 2.x, not for python 3 as you asked. I've updated the code to work for python 3. – jcollado Jan 16 '12 at 13:46
this looks promising!!! My next question (please bare with me, it's my second day of python) is how do I then access info from each dictionary. Do I need to give them unique names? Say I want to cross reference the date with a spreadsheet. – beoliver Jan 16 '12 at 13:50
@user969617 There's no need to give a name to the dictionaries. You can store them in a list and access the list by index and the dictionary by key: `rows[0]['name']` – jcollado Jan 16 '12 at 14:23
Thanks man your code works perfectly well for parsing hadoop fs -ls as well!!!!!!!!! – Bohdan Oct 01 '13 at 23:35

score 7 · Answer 3 · answered Jan 16 '12 at 13:19

7

If what you really want is to list a directory, rather use os.listdir

import os
files = os.listdir('/path/to/dir')
for file in files:
    print(file)

answered Jan 16 '12 at 13:19

Gary van der Merwe

9,134
3
49
80

i actually want to list the airport tables, but thought that I would use a simple example. The actual output is going to be `call(['/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport', '-s'])` – beoliver Jan 16 '12 at 13:29

score 2 · Answer 4 · edited May 23 '17 at 12:08

Read about Popen. the set you asked for you get with

import subprocess
proc = subprocess.Popen(['ls','-l'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
myset=set(proc.stdout)

or do something like

for x in proc.stdout : print x

and the same for stderr

you can examine the state of the process with

proc.poll()

or wait for it to terminate with

proc.wait()

python subprocess output to list or file

6 Answers6

Linked