Need to read string into a float array

Question

I have a text file like below. I want to read given values as a float list. After that I am going to do some calculations. I used split function and convertion to float. But I cannot convert first one and last one because those two has square brackets. ([ ]). it gave an error as below.

File format

[-1.504, 1.521, 1.531, 1.1579, -2.2976, 2.5927,... 1000 records]
[2.758, -0.951, -1.7952, 0.4255, 2.5403, 1.0233,... 1000 records]
[0.682, -2.205, 2.1981, 2.1329, 0.1574, -0.4695,... 1000 records]

Error

 Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
 ValueError: could not convert string to float: [0.682

Code I used

F = open('XYZ.txt', 'r')
>>> for line in F:
...     P = line.split(',')
...     P

Can any one give me an idea how do I read that values into a float array like below.

X = [-1.504, 1.521, 1.531, 1.1579, -2.2976, 2.5927,... 1000 records]
Y = [2.758, -0.951, -1.7952, 0.4255, 2.5403, 1.0233,... 1000 records]
Z = [0.682, -2.205, 2.1981, 2.1329, 0.1574, -0.4695,... 1000 records]

Then I can call values like X[1], X[999]

Answer 1

Use ast.literal_eval() to parse each line into a list of floats:

import ast

with open('XYZ.txt', 'r') as infh:
    for line in infh:
        row = ast.literal_eval(line)
        print row

The ast.literal_eval() interprets each line as containing literal Python values, supporting lists, tuples, dictionaries, strings and number literals.

Demo:

>>> import ast
>>> ast.literal_eval('[-1.504, 1.521, 1.531, 1.1579, -2.2976, 2.5927]\n')
[-1.504, 1.521, 1.531, 1.1579, -2.2976, 2.5927]

Answer 2

you can just eval them:

for line in F:
  my_arr = eval(line)

Answer 3

Use json.loads():

import json
line = "[-1.504, 1.521, 1.531, 1.1579, -2.2976]"
row = json.loads(line)

Edit: I found @Martijn answer interesting as I almost never uses ast, so I ran a small benchmark:

import timeit
import random

line = [random.random() for x in range(1000)]
n = 10000

json_setup = 'line = "{}"; import json'.format(line)
json_work = 'json.loads(line)'
json_time = timeit.timeit(json_work, json_setup, number=n)
print "json: ", json_time

ast_setup = 'line = "{}"; import ast'.format(line)
ast_work = 'ast.literal_eval(line)'
ast_time = timeit.timeit(ast_work, ast_setup, number=n)
print "ast: ", ast_time

print "time ratio ast/json: ", ast_time / json_time

I ran this code several times and consistently got this kind of results:

$ python json-ast-bench.py 
json: 4.3199338913
ast: 28.4827561378
time ratio ast/json:  6.59333148483

So it appears that ast is almost an order of magnitude slower for this use case.

Edit: Why is json.loads an order of magnitude faster than ast.literal_eval?