0

My python code produces a pandas dataframe that looks as follows:

enter image description here

I need to transform it to another format to achieve following: loop through every row in the dataframe and output as many data frames as rows in the table. Each dataframe should have a additional column: timestamp and be named as the value in "Type" Column. So for instance I'd have

enter image description here

I am struggling with where to start- I hope someone here can advise me?

Agatella
  • 11
  • 2
  • 1
    Please [do not post images](https://meta.stackoverflow.com/questions/285551/why-not-upload-images-of-code-errors-when-asking-a-question) of your data. You can include [code that creates a dataframe or the output of `print(df)`](https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples) (or of a few rows and columns that allow to reproduce the example) – Cimbali Jun 21 '21 at 09:50
  • 1
    Noted for the future – Agatella Jun 22 '21 at 14:12

2 Answers2

0

Here is a code for what you want to achieve. It takes a csv file like yours. Loops through the rows. Adds a column with current time and saves each row in a separate csv. Let me know if it works for you.

import pandas as pd 
from datetime import datetime
#Give path to your csv
df = pd.read_csv('C:/Users/username/Downloads/test.csv')

#iterating on rows in dataframe
for index, row in df.iterrows():

   #adding a new columns with value in the row
   df.loc[index, 'Timestamp'] = datetime.now().strftime('%c')
   print(df.loc[index])

   #saving row in a new dataframe
   df_new = df.loc[index].to_frame().T

   #saving the dataframe in a separate csv
   df_new.to_csv(f'C:/Users/username/Downloads/test_{index}.csv', index= False)
sheharbano
  • 211
  • 2
  • 13
  • Thanks @sheharbano for quick help! What I see is when I print(df.loc[index]) when I see all the elements separately but then you use df_new = df.loc[index].to_frame().T where df_new contains only the data frame created from the last row (last indexed element). I assume I'd need to loop through the indexed rows to output as many data frames as there are rows? – Agatella Jun 21 '21 at 11:43
  • It is looping on rows and already producing as many dataframes as the number of rows. You can check the csv files created at the end. – sheharbano Jun 21 '21 at 11:59
  • It doesn't print the whole dataframe on printing df.loc[index], it just prints a single row but vertically. In the next line I am transposing it to view it horizontally. – sheharbano Jun 21 '21 at 12:08
0

Pandas' .to_records? is your friend (doc here.)

from datetime import datetime

list_of_final_dataframes = []
for record in df.to_dict(orient='records'):
    record_with_timestamp = {**record, **{'timestamp': datetime.now()}}
    list_of_final_dataframes.append(pd.DataFrame([record_with_timestamp]))
Campello
  • 325
  • 1
  • 3
  • 10