How to read in dataframe of lists from csv

Asked Oct 28 '21 at 05:29

Active Oct 28 '21 at 06:11

Viewed 33 times

Problem

Given:

df = pd.DataFrame([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
df = df.applymap(lambda x: [x, 0])

        0       1       2
0  [1, 0]  [2, 0]  [3, 0]
1  [4, 0]  [5, 0]  [6, 0]
2  [7, 0]  [8, 0]  [9, 0]

df.to_csv("list_entries.csv")

How do you read the data back in, preserving the original data types?

What I've Tried

This answer mentions use of converters, but I can't figure out how to apply it to all columns.

I've tried instead to use applymap():

import ast
df.applymap(lambda x: ast.literal_eval(x))

but get this error:

NameError: name 'ast' is not defined

(Follow up to this question.)

Edit:
From my debugger:

(Pdb) df.apply(lambda x: ast.literal_eval(x))
*** NameError: name 'ast' is not defined
(Pdb) ast
<module 'ast' from '/usr/lib/python3.9/ast.py'>

Edit:
Oddly it works fine outside the debugger.

edited Oct 28 '21 at 06:11

asked Oct 28 '21 at 05:29

young_souvlaki

Use `import ast` – jezrael Oct 28 '21 at 05:33
1

With nested data, I recommend using `to_pickle` and `read_pickle`. Or you can use `to_json` and `read_json` if you insist on text data. – Quang Hoang Oct 28 '21 at 05:40
1

no need to import `ast` `df['0'] = df['0'].apply(lambda x: eval(x))` – Danial Shabbir Oct 28 '21 at 06:07
@DanialShabbir https://stackoverflow.com/questions/15197673/using-pythons-eval-vs-ast-literal-eval – young_souvlaki Oct 28 '21 at 06:12
@DanialShabbir That noted, yea I could get away with less dependencies. – young_souvlaki Oct 28 '21 at 06:14

0 Answers0