0

I'm doing sentiment analysis of Tweeter data. For this work, I've made some datasets in CSV format where different month in different dataset. When I do the preprocessing of every dataset individually, I want to save all dataset in 1 single CSV file. but when I write the below's code by using pandas dataframe:

df.to_csv('dataset.csv', index=False)

It removes previous data (Rows) of that dataset. Is there any way that I can keep the previous data too on that file? So that I can merge all data together. Thank you..........

rabby26
  • 13
  • 3
  • like this: https://stackoverflow.com/questions/17530542/how-to-add-pandas-data-to-an-existing-csv-file ? – YevKad Jan 16 '21 at 14:16

2 Answers2

0

It's not entirely clear what you want from your question, so this is just a guess, but something like this might be what you're looking for. if you keep assigning dataframes to df, then new data will overwrite the old data. Try reassigning them to differently named dataframes like df1 and `df21. Then you can merge them.

# vertically merge the multiple dataframes and reassign to new variable
df = pd.concat([df1, df2])
# save the dataframe 
df.to_csv('my_dataset.csv', index=False)
thehand0
  • 1,123
  • 4
  • 14
0

In python you can use the open("file") method with the parameter 'a': open("file", 'a'). The 'a' means "append" so you will add lines at the end of your file. You can use the same parameter for the pandas.dataFrame.to_csv() method. e.g:

import pandas as pd

# code where you get data and return df

pd.df.to_csv("file", mode='a')

@thehand0: Your code works, but it's inefficient, so it will take longer for your script to run.

Saveli Frontzek
  • 1
  • 2