DataFrame save at each iteration

Question

I am trying to save some values from my training model as DataFrame which works okay but it only save a data for the first loop and it stops. I have tried to put it in a For loop but didn't help. What ways is available for me please to save for the whole training?

 df=pd.DataFrame({'Image_Name':names_preds, 'Target':targets.detach(), 'Prediction':preds.detach()})
            print(df)
            df.to_csv(r'test5.csv', index=False)

can you please include the entire script? Its clear you're using pandas and numpy, is there anything else you are importing/using? — samman, Jul 08 '20 at 23:46
https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples — Paul H, Jul 08 '20 at 23:54
We don't need all of the pytorch, ML, and CLI stuff. mock up a dataframe and show us what you're trying to do — Paul H, Jul 08 '20 at 23:55
I have included the code concerning the DataFrame only. I need to save those values at each iteration. At the moment the DF only save one iteration. — AA2018-2020, Jul 09 '20 at 00:06
Read through the link i posted above. it's not at all clear what's going on. — Paul H, Jul 09 '20 at 00:20
Sorry not sure how i could duplicate the issue i am having. As i said I am trying to save the values in my dataframe at each iteration. — AA2018-2020, Jul 09 '20 at 00:33

score 0 · Accepted Answer · answered Jul 09 '20 at 01:22

0

Adding mode='a' sorted the issue.

df=pd.DataFrame({'Image_Name':names_preds, 'Target':targets.detach(), 'Prediction':preds.detach()})
            print(df)
            df.to_csv(r'test5.csv',mode='a', index=False)

answered Jul 09 '20 at 01:22

AA2018-2020

341
1
6
16

DataFrame save at each iteration

1 Answers1