How do you read a file into a list in Python?

Question

I want to prompt a user for a number of random numbers to be generated and saved to a file. He gave us that part. The part we have to do is to open that file, convert the numbers into a list, then find the mean, standard deviation, etc. without using the easy built-in Python tools.

I've tried using open but it gives me invalid syntax (the file name I chose was "numbers" and it saved into "My Documents" automatically, so I tried open(numbers, 'r') and open(C:\name\MyDocuments\numbers, 'r') and neither one worked).

Answer 1

with open('C:/path/numbers.txt') as f:
    lines = f.read().splitlines()

this will give you a list of values (strings) you had in your file, with newlines stripped.

also, watch your backslashes in windows path names, as those are also escape chars in strings. You can use forward slashes or double backslashes instead.

Answer 2

Two ways to read file into list in python (note these are not either or) -

1. use of with - supported from python 2.5 and above
2. use of list comprehensions

1. use of with

This is the pythonic way of opening and reading files.

#Sample 1 - elucidating each step but not memory efficient
lines = []
with open("C:\name\MyDocuments\numbers") as file:
    for line in file: 
        line = line.strip() #or some other preprocessing
        lines.append(line) #storing everything in memory!

#Sample 2 - a more pythonic and idiomatic way but still not memory efficient
with open("C:\name\MyDocuments\numbers") as file:
    lines = [line.strip() for line in file]

#Sample 3 - a more pythonic way with efficient memory usage. Proper usage of with and file iterators. 
with open("C:\name\MyDocuments\numbers") as file:
    for line in file:
        line = line.strip() #preprocess line
        doSomethingWithThisLine(line) #take action on line instead of storing in a list. more memory efficient at the cost of execution speed.

the .strip() is used for each line of the file to remove \n newline character that each line might have. When the with ends, the file will be closed automatically for you. This is true even if an exception is raised inside of it.

2. use of list comprehension

This could be considered inefficient as the file descriptor might not be closed immediately. Could be a potential issue when this is called inside a function opening thousands of files.

data = [line.strip() for line in open("C:/name/MyDocuments/numbers", 'r')]

Note that file closing is implementation dependent. Normally unused variables are garbage collected by python interpreter. In cPython (the regular interpreter version from python.org), it will happen immediately, since its garbage collector works by reference counting. In another interpreter, like Jython or Iron Python, there may be a delay.

Answer 3

f = open("file.txt")
lines = f.readlines()

Look over here. readlines() returns a list containing one line per element. Note that these lines contain the \n (newline-character) at the end of the line. You can strip off this newline-character by using the strip()-method. I.e. call lines[index].strip() in order to get the string without the newline character.

As joaquin noted, do not forget to f.close() the file.

Converting strint to integers is easy: int("12").

Answer 4

The pythonic way to read a file and put every lines in a list:

from __future__ import with_statement #for python 2.5
with open('C:/path/numbers.txt', 'r') as f:
    lines = f.readlines()

Then, assuming that each lines contains a number,

numbers =[int(e.strip()) for e in lines]

Answer 5

You need to pass a filename string to open. There's an extra complication when the string has \ in it, because that's a special string escape character to Python. You can fix this by doubling up each as \\ or by putting a r in front of the string as follows: r'C:\name\MyDocuments\numbers'.

Edit: The edits to the question make it completely different from the original, and since none of them was from the original poster I'm not sure they're warrented. However it does point out one obvious thing that might have been overlooked, and that's how to add "My Documents" to a filename.

In an English version of Windows XP, My Documents is actually C:\Documents and Settings\name\My Documents. This means the open call should look like:

open(r"C:\Documents and Settings\name\My Documents\numbers", 'r')

I presume you're using XP because you call it My Documents - it changed in Vista and Windows 7. I don't know if there's an easy way to look this up automatically in Python.

Answer 6

To summarize a bit from what people have been saying:

f=open('data.txt', 'w') # will make a new file or erase a file of that name if it is present
f=open('data.txt', 'r') # will open a file as read-only
f=open('data.txt', 'a') # will open a file for appending (appended data goes to the end of the file)

If you wish have something in place similar to a try/catch

with open('data.txt') as f:
    for line in f:
        print line

I think @movieyoda code is probably what you should use however

Answer 7

hdl = open("C:/name/MyDocuments/numbers", 'r')
milist = hdl.readlines()
hdl.close()

Answer 8

If you have multiple numbers per line and you have multiple lines, you can read them in like this:

    #!/usr/bin/env python

    from os.path import dirname

    with open(dirname(__file__) + '/data/path/filename.txt') as input_data:
        input_list= [map(int,num.split()) for num in input_data.readlines()]