Save the data in CSV after every update

Question

Hi I have some data I want to save it in dataframe after every update. but It always override my previous data. is there any method to keep my previous data save and add new to it.

df = pd.DataFrame(columns=['Entry','Middle','Exit'])
def function():
    entry_value = 178.184 # data comming from server
    middle_value = 14.121 # data comming from server
    exit_value = 19.21 # data comming from server
    df1 = df.append({'Entry' : entry_value , 'Middle' : middle_value, 'Exit' : exit_value}, ignore_index = True)
    df1.to_csv('abc.csv')
 
i = 0
while i < 5:
    function()
    i += 1

this entry_value, middle_value and exit_value is change. sometime it's not change. in this example i want that my csv have same data 5 times

Note:: here the value is hard codded but it's comming from server but in this format

Answer 1

You can use concat function (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.concat.html)

for example:

import pandas as pd

df = pd.DataFrame(columns=['Entry','Middle','Exit'])
def function():
    global df
    entry_value = 178.184 # data comming from server
    middle_value = 14.121 # data comming from server
    exit_value = 19.21 # data comming from server'
    
    new_row = pd.DataFrame.from_dict([{'Entry' : entry_value , 'Middle' : middle_value, 'Exit' : exit_value}], orient='columns')
    df = pd.concat([df, new_row])
    df.to_csv('abc.csv')
 
i = 0
while i < 5:
    function()
    i += 1

Also if you want to have every version of your CSV file you can add a counter to the end of your CSV file name. for example:

import pandas as pd

df = pd.DataFrame(columns=['Entry','Middle','Exit'])
def function(n):
    global df
    entry_value = 178.184 # data comming from server
    middle_value = 14.121 # data comming from server
    exit_value = 19.21 # data comming from server'
    
    new_row = pd.DataFrame.from_dict([{'Entry' : entry_value , 'Middle' : middle_value, 'Exit' : exit_value}], orient='columns')
    df = pd.concat([df, new_row])
    df.to_csv(f'abc{n}.csv')
 
i = 0
while i < 5:
    function(i)
    i += 1

Answer 2

This answer might be able to help you much better by giving you clarity on multiple ways of appending the data in dataframe.