Save pandas groups to separate CSV files

Question

Is it possible to return multiple df's to independent csv's without using .to_csv. Using the code below I'm manually returning desired values and exporting those to a csv. This is ok if I only have a few files to export but if there's numerous files or file names that constantly change over datasets it can be cumbersome.

Is there a more efficient way to return desired values if you have a specific value list and export those to a csv?

import pandas as pd

d = ({
    'C' : ['08:00:00','XX','08:10:00','XX','08:41:42','XX','08:50:00','XX', '09:00:00', 'XX','09:15:00','XX','09:21:00','XX','09:30:00','XX','09:40:00','XX'],
    'D' : ['Home','Home','Home','Home','Away','Away','Shops','Shops','Away','Away','Shops','Shops','Home','Home','Away','Away','Home','Home'],
    'E' : ['Num:','','Num:','','Num:','','Num:','','Num:', '','Num:','','Num:','','Num:', '','Num:', ''],
    'F' : ['1','','1','','1','','1','','1', '','2','','2','','1', '','2',''],   
    'A' : ['A','','A','','A','','A','','A','','A','','A','','A','','A',''],           
    'B' : ['Stop','','Res','','Stop','','Start','','Res','','Stop','','Res','','Start','','Start','']
    })

df = pd.DataFrame(data=d)

#List of designated places
values = ['Home', 'Away', 'Shops']

#Export to csv
Home = df.loc[df['D'] == 'Home'].to_csv('Home.csv')
Away = df.loc[df['D'] == 'Away'].to_csv('Away.csv')
Shops = df.loc[df['D'] == 'Shops'].to_csv('Shops.csv')

Sorry, fixed the duplicate link. You may want something like this: https://stackoverflow.com/questions/45871314/split-dataframe-into-a-dictionary-of-groups-from-multiple-columns/45871429#45871429 — cs95, Jun 13 '18 at 04:45

cs95 · Accepted Answer · 2019-04-14T18:24:00.317

5

Filter with isin, then perform a groupby on "D" and iteratively save to CSV.

incl = ['Home', 'Away', 'Shops']    
for k, g in df[df['D'].isin(incl)].groupby('D'):
    g.to_csv(f'{k}.csv')  # '{}.csv'.format(k)

The isin filtering step is important if and only if you have more categories than what you want to save. If this isn't the case and you want to save everything, your solution will simplify:

for k, g in df.groupby('D'):
    ...

edited Apr 14 '19 at 18:24

answered Jun 13 '18 at 04:46

cs95

379,657
97
704
746

Thanks @coldspeed – Jun 13 '18 at 06:29
This will probably have to be another question but could I add new columns before exporting? – Jun 13 '18 at 06:32
@PeterJames123 it depends on what the columns are you want to add. Anything you need to do can be done inside the for loop. – cs95 Jun 13 '18 at 06:58
Thanks @coldspeed. I got it half working https://stackoverflow.com/q/50831061/9394674 – Jun 13 '18 at 07:11

Save pandas groups to separate CSV files

1 Answers1