Python read text files written in columns into lists

Question

A file hoge.txt which I want to read is as below

x1 y1 z1 
x2 y2 z2
x3 y3 z3

my code is now

lines = open('hoge.txt').readlines()
line_parts = [line.split() for line in lines]
x  = [float(v[0]) for v in line_parts]
y  = [float(v[0]) for v in line_parts]
z  = [float(v[0]) for v in line_parts]

Can I write this flow easier and simpler?

Answer 1

• You can use the context manger statement with.

• I'd actually question whether you need three separate variables to hold the values from each line. Just use a 2d list to hold the values from each line of your file.

• Your code has a slight bug. You need to access the 1st index of each value in each line of your file.¹

Here is your code with the above modifications:

with open('hoge.txt') as file:
    lines = file.readlines()
    data = [[float(val[1]) for val in line.split()] for line in lines]
# ... use `data`

---

¹_{I'm assuming, of course, that the values in your file are not places-holders. If they are, then doing float(val) is all you need.}

Answer 2

There are several things you can do to improve on the source you mentioned. As a disclaimer, I'm assuming the x1, x2, etc... in your file are placeholders for numeric values, since otherwise you'd probably be better off without messing with a file at all. If that's a bad assumption, please let me know so I can edit my answer.

1. File objects are actually iterable, and there is no need to call read() or readlines().
2. Use of a context manager with the keyword with allows you to ensure the file is closed when you're done with it.
3. I've always thought it was a neat trick that you could use zip(*[...]) to transpose rows and columns.
4. Using multiple assignment, you can get the explicit x, y, and z variables you want.
5. The way the question was originally asked, x, y, and z would have had identical contents and ignored most of the file you were reading. Assuming this was undesired behavior, I fixed the bug.

In summary, you get the following source code:

with open('hoge.txt') as file:
    x, y, z = zip(*[[float(el) for el in line] for line in file])

It's probably worth mentioning that for just about anything you might want to do with that data, the module pandas is probably the tool you want to use. You could use the read_csv() function to read the file, and if you wanted the rows and columns transposed as in your example code, you could use the .T property on the resulting DataFrame object.

Additionally, part of the reason your code is verbose is that the file doesn't naturally align with how you want to use it. If you have such flexibility and don't need the file for anything else, a more natural format would be

x1 x2 x3
y1 y2 y3
z1 z2 z3

And then you could get away with slightly less verbose code.

with open('hoge.txt') as file:
    x, y, z = [[float(el) for el in line] for line in file]

Lastly, for simple typecasting of everything in a list, the numpy module is a joy to work with (after a medium learning curve) and admits the astype() method on its array objects. Alternatively, I find the map() function to be a little easier to read in such circumstances if you want to work with raw python lists. Since you're using Python 3, you would need to import map().

from itertools import map

with open('hoge.txt') as file:
    x, y, z = zip(*[map(float, line) for line in file])