Put maximum of each column from a list of dataframe into new dataframe

Question

I recently started working with pandas dataframes.

I have a list of dataframes called 'arr'.

Edit: All the dataframes in 'arr' have same columns but different data. Also, I have an empty dataframe 'ndf' which I need to fill in using the above list. How do I iterate through 'arr' to fill in the max values of a column from 'arr' into a row in 'ndf' So, we'll have

Number of rows in ndf = Number of elements in arr

I'm looking for something like this:

columns=['time','Open','High','Low','Close']
ndf=DataFrame(columns=columns)
ndf['High']=arr[i].max(axis=0)

Do all of your dataframes in the list have the same columns? — James, Jun 15 '15 at 11:35
Could you please give a small example of what data you have and what you want in the end? — firelynx, Jun 15 '15 at 11:54
@James, Yes the all the dataframes in the list have same columns with different data. — AmanArora, Jun 16 '15 at 05:28

score 0 · Answer 1 · edited May 23 '17 at 12:24

Based on your description, I assume a basic example of your data looks something like this:

import pandas as pd

data =[{'time':'2013-09-01','open':249,'high':254,'low':249,'close':250},
       {'time':'2013-09-02','open':249,'high':256,'low':248,'close':250}]
data2 =[{'time':'2013-09-01','open':251,'high':253,'low':248,'close':250},
        {'time':'2013-09-02','open':245,'high':251,'low':243,'close':247}]
df = pd.DataFrame(data)
df2 = pd.DataFrame(data2)
arr = [df, df2]

If that's the case, then you can simply iterate over the list of dataframes (via enumerate()) and the columns of each dataframe (via iteritems(), see http://pandas.pydata.org/pandas-docs/stable/basics.html#iteritems), populating each new row via a dictionary comprehension: (see Create a dictionary with list comprehension in Python):

ndf = pd.DataFrame(columns = df.columns)
for i, df in enumerate(arr):
    ndf = ndf.append(pd.DataFrame(data = {colName: max(colData) for colName, colData in df.iteritems()}, index = [i]))

If some of your dataframes have any additional columns, the resulting dataframe ndf will have NaN entries in the relevant places.

Put maximum of each column from a list of dataframe into new dataframe

1 Answers1