Saving DataFrame names as .csv file names in Pandas

Question

In [37]: blue = pd.DataFrame({'A': ['foo','foo','foo','bar','bar'], 'B': [4.0, 4.0, 5.0, 8.0, 8.0]})

In [38]: blue
Out[38]: 
     A  B
0  foo  4
1  foo  4
2  foo  5
3  bar  8
4  bar  8

In [39]: red = pd.DataFrame({'A': ['foo','foo','foo','bar','bar'], 'B': [np.nan, np.nan, np.nan, np.nan, np.nan]})

In [40]: red
Out[40]: 
     A   B
0  foo NaN
1  foo NaN
2  foo NaN
3  bar NaN
4  bar NaN

In [41]: for df in [blue, red]:
   ....:     df.to_csv(str(df))
   ....:     

In [42]: !ls
     A  B?0  foo  4?1  foo  4?2  foo  5?3  bar  8?4  bar  8       A   B?0  foo NaN?1  foo NaN?2  foo NaN?3  bar NaN?4  bar NaN  postinstall.sh  vagrant

I have some DataFrames. I loop over each DataFrame to work on them. At the end of the loop I want to save each DataFrame as a .csv file named after the DataFrame. I know that it's generally difficult to stringify the name of a variable in Python, but I have to think that I'm missing something obvious here. There is no "name" attribute for DataFrames, so what do I do?

Why not hold them in a dictionary `{'red': pd.DataFrame(...), ...}` then save them using the key as filename? — jonrsharpe, Aug 15 '14 at 19:40

score 17 · Accepted Answer · edited May 23 '17 at 12:05

17

You can just add an attribute to the df, same as any other python object that has a __dict__ attribute and use it later:

In [2]:

blue.name = 'blue'
red.name = 'red'
df_list = [blue, red]
for df in df_list:
    print(df.name)
    df.to_csv(df.name + '.csv')
blue
red

Even better, for convenience you can store the csv name and use it later too:

In [5]:

blue.name = 'blue'
blue.csv_path = 'blue.csv'
red.name = 'red'
red.csv_path = 'red.csv'
df_list = [blue, red]
for df in df_list:
    print(df.name)
    print(df.csv_path)
    df.to_csv(df.csv_path)
blue
blue.csv
red
red.csv

EDIT As @Jeff has pointed out, the attributes will not persist across most operations on the df as a copy of the df is returned and these attributes are not copied across so be aware of this.

edited May 23 '17 at 12:05

Community

1
1

answered Aug 15 '14 at 19:47

EdChum

376,765
198
813
562

Thanks EdChum. I always seem to forget how versatile a dictionary-like object can be. – verbsintransit Aug 15 '14 at 20:17
5

one note: the attributes will NOT persist across operations, so if you do almost any operation, e.g. ``df[df.A>0]`` for example you are getting a NEW frame. – Jeff Aug 15 '14 at 20:32
Thanks for the note @Jeff. As it happens it does work for me, but that's definitely good to know. – verbsintransit Aug 15 '14 at 22:53
@I_m_LeMarque don't post an error to me as a comment, post an actual new question. It's counter-productive to ask coding questions using comments, we require data, your code, desired result and all errors – EdChum Nov 24 '17 at 10:35
@I_m_LeMarque again, I'm not answering questions regarding your situation using comments, post an actual new question. I will ignore any further comments on this – EdChum Nov 24 '17 at 10:50

Saving DataFrame names as .csv file names in Pandas

1 Answers1