2

I have a Dataframe (called df) that has list of tickets worked for a given date. I have a script that runs each day where this df gets generated and I would like to have a new master dataframe (lets say df_master) that appends values form df to a new Dataframe. So anytime I view df_master I should be able to see all the tickets worked across multiple days. Also would like to have a new column in df_master that shows date when the row was inserted.

Given below is how df looks like:

1001
1002
1003
1004

I tried to perform concat but it threw an error 

TypeError: first argument must be an iterable of pandas objects, you passed an object of type "Series"

Update

df_ticket = tickets['ticket']
df_master = df_ticket
df_master['Date'] = pd.Timestamp('now').normalize()
L = [df_master,tickets] 
master_df = pd.concat(L)
master_df.to_csv('file.csv', mode='a', header=False, index=False)
dark horse
  • 447
  • 1
  • 6
  • 17
  • What did you try so far? also please post a sample output. for appending dfs generated in a loop see [this](https://stackoverflow.com/questions/28669482/appending-pandas-dataframes-generated-in-a-for-loop) – anky Dec 21 '18 at 07:16

1 Answers1

2

I think you need pass sequence to concat, obviously list is used:

objs : a sequence or mapping of Series, DataFrame, or Panel objects

If a dict is passed, the sorted keys will be used as the keys argument, unless it is passed, in which case the values will be selected (see below). Any None objects will be dropped silently unless they are all None in which case a ValueError will be raised

L = [s1,s2] 
df = pd.concat(L)

And it seems you pass only Series, so raised error:

df = pd.concat(s)

For insert Date column is possible set pd.Timestamp('now').normalize(), for master df I suggest create one file and append each day DataFrame:

df_ticket = tickets[['ticket']]
df_ticket['Date'] = pd.Timestamp('now').normalize()
df_ticket.to_csv('file.csv', mode='a', header=False, index=False)

df_master = pd.read_csv('file.csv',  header=None)
jezrael
  • 822,522
  • 95
  • 1,334
  • 1,252
  • thanks for the help. I have one issue however. I see the time current date gets added to the same column as the ticket. I am unable to add a new column to the Dataframe df_master.. – dark horse Dec 21 '18 at 08:14
  • @darkhorse - I think `df_master` with `concat` is not necessary, only add column with date and append to file. And if need all data use `df_master = pd.read_csv('file.csv', header=None)`, check edited answer. – jezrael Dec 21 '18 at 08:20
  • I tried the above and it still does the same. Adds the timestamp as a new row and not as a column. – dark horse Dec 21 '18 at 08:27
  • 1
    @darkhorse - you are right, need `df_ticket = tickets[['ticket']]` instaed `df_ticket = tickets['ticket']` for one column DataFrame – jezrael Dec 21 '18 at 08:28