Reading a text document containing python list into a python program

Question

I have a text file(dummy.txt) which reads as below:

['abc',1,1,3,3,0,0]
['sdf',3,2,5,1,3,1]
['xyz',0,3,4,1,1,1]

I expect this to be in lists in python as below:

article1 = ['abc',1,1,3,3,0,0]
article2 = ['sdf',3,2,5,1,3,1]
article3 = ['xyz',0,3,4,1,1,1]

That many articles have to be created as many lines present in dummy.txt

I was trying the following things: Opened the file, split it by '\n' and appended it to an empty list in python, it had extra quotes and square brackets hence tried to use 'ast.literal_eval' which did not work as well.

my_list = []
fvt = open("dummy.txt","r")
for line in fvt.read():
    my_list.append(line.split('\n'))
    my_list = ast.literal_eval(my_list)

I also tried to manually remove additional quotes and extra square brackets using replace, that did not help me either. Any leads much appreciated.

Answer 1

This should help.

import ast

myLists = []
with open(filename) as infile:
    for line in infile:                         #Iterate Each line
        myLists.append(ast.literal_eval(line))  #Convert to python object and append.
print(myLists)  

Output:

[['abc', 1, 1, 3, 3, 0, 0], ['sdf', 3, 2, 5, 1, 3, 1], ['xyz', 0, 3, 4, 1, 1, 1]]

Answer 2

fvt.read() will produce the entire file string, so that means line will contain a single character string. So this will not work very well, you also use literal_eval(..) with the entire list of strings, and not a single string.

You can obtain the results by iterating over the file handler, and each time call literal_eval(..) on a single line:

from ast import literal_eval

with open("dummy.txt","r") as f:
    my_list = [literal_eval(line) for line in f]

or by using map:

from ast import literal_eval

with open("dummy.txt","r") as f:
    my_list = list(map(literal_eval, f))

We then obtain:

>>> my_list
[['abc', 1, 1, 3, 3, 0, 0], ['sdf', 3, 2, 5, 1, 3, 1], ['xyz', 0, 3, 4, 1, 1, 1]]

Answer 3

ast.literal_eval is the right approach. Note that creating a variable number of variables like article1, article2, ... is not a good idea. Use a dictionary instead if your names are meaningful, a list otherwise.

As Willem mentioned in his answer fvt.read() will give you the whole file as one string. It is much easier to exploit the fact that files are iterable line-by-line. Keep the for loop, but get rid of the call to read.

Additionally,

my_list = ast.literal_eval(my_list)

is problematic because a) you evaluate the wrong data structure - you want to evaluate the line, not the list my_list to which you append and b) because you reassign the name my_list, at this point the old my_list is gone.

Consider the following demo. (Replace fake_file with the actual file you are opening.)

>>> from io import StringIO
>>> from ast import literal_eval
>>> 
>>> fake_file = StringIO('''['abc',1,1,3,3,0,0]
... ['sdf',3,2,5,1,3,1]
... ['xyz',0,3,4,1,1,1]''')
>>> result = [literal_eval(line) for line in fake_file]
>>> result
[['abc', 1, 1, 3, 3, 0, 0], ['sdf', 3, 2, 5, 1, 3, 1], ['xyz', 0, 3, 4, 1, 1, 1]]

Of course, you could also use a dictionary to hold the evaluated lines:

>>> result = {'article{}'.format(i):literal_eval(line) for i, line in enumerate(fake_file, 1)}
>>> result
{'article2': ['sdf', 3, 2, 5, 1, 3, 1], 'article1': ['abc', 1, 1, 3, 3, 0, 0], 'article3': ['xyz', 0, 3, 4, 1, 1, 1]}

where now you can issue

>>> result['article2']
['sdf', 3, 2, 5, 1, 3, 1]

... but as these names are not very meaningful, I'd just go for the list instead which you can index with 0, 1, 2, ...

Answer 4

When I do this:

import ast
x = '[ "A", 1]'
x = ast.literal_eval(x)
print(x)

I get:

 ["A", 1]

So, your code should be:

for line in fvt.read():
    my_list.append(ast.literal_eval(line))

Answer 5

Try this split (no imports needed) (i recommend):

with open('dummy.txt','r') as f:
    l=[i[1:-1].strip().replace("'",'').split(',') for i in f]

Now:

print(l)

Is:

[['abc', 1, 1, 3, 3, 0, 0], ['sdf', 3, 2, 5, 1, 3, 1], ['xyz', 0, 3, 4, 1, 1, 1]]

As expected!!!