Python: group by sum with condition

Question

I have a dataframe like the following:

df = pd.DataFrame({'condition' : ['a','b','b','b','a','a'],
               'name' : ['one', 'one', 'two', 'three', 'three', 'three'],
               'data1' : [7, 3, 48, 13, 27, 12]})
df
  condtion  data1   name
0   a          7    one
1   b          3    one
2   b         48    two
3   b         13    three
4   a         27    three
5   a         12    three

For each name I want to sum on data1 and use information of condition=a if I have that information, condition=b otherwise. At the end I would like a dataframe like the following:

df1 
    name   total
0   one      7
1   two     48
2   three   39

Please refer https://stackoverflow.com/questions/17266129/python-pandas-conditional-sums — Suresh Raju, Jul 02 '18 at 12:41

jezrael · Accepted Answer · 2018-07-02T12:54:28.513

You can aggregate groupby with aggregate sum and reshape by unstack, last replace NaNs for missing catagories a by fillna:

df = df.groupby(['name','condition'], sort=False)['data1'].sum().unstack()
df['total'] = df['a'].fillna(df['b'])
print (df)
condition     a     b  total
name                        
one         7.0   3.0    7.0
two         NaN  48.0   48.0
three      39.0  13.0   39.0

And for new DataFrame:

df1 = df.reset_index().rename_axis(None, 1)[['name','total']]
print (df1)
    name  total
0    one    7.0
1    two   48.0
2  three   39.0

Another solution with apply:

def f(x):
    if (x['condition'] == 'a').any():
        return x.loc[x['condition'] == 'a', 'data1'].sum() 
    else:
        return x.loc[x['condition'] == 'b', 'data1'].sum()

df1 = df.groupby('name', sort=False).apply(f).reset_index(name='total')
print (df1)
    name  total
0    one      7
1    two     48
2  three     39

Better one is create Series for aggregating filtered DataFrame and then combine_first, but this solution omit all names groups with no a or b conditions:

a = df.loc[df['condition'] == 'a'].groupby('name', sort=False)['data1'].sum()
b = df.loc[df['condition'] == 'b'].groupby('name', sort=False)['data1'].sum()

df = a.combine_first(b).reset_index(name='total')
print (df)
    name  total
0    one    7.0
1  three   39.0
2    two   48.0

score 0 · Answer 2 · answered Jul 02 '18 at 12:50

You can groupby name and condition to find the sum of data1 and then sort_values by name and condition before selecting first entry for each group in grouping by name:

df.groupby(['name', 'condition'])['data1'].sum().reset_index(name='total').sort_values(['name','condition']).groupby(['name']).first().reset_index()[['name', 'total']]

Output:

    name    total
0   one     7
1   three   39
2   two     48

score 0 · Answer 3 · answered Jul 02 '18 at 13:00

You can use pd.pivot_table with aggfunc='sum':

df = pd.DataFrame({'condition' : ['a','b','b','b','a','a'],
                   'name' : ['one', 'one', 'two', 'three', 'three', 'three'],
                   'data1' : [7, 3, 48, 13, 27, 12]})

res = df.pivot_table(index='name', columns='condition', values='data1', aggfunc='sum')

condition     a     b
name                 
one         7.0   3.0
three      39.0  13.0
two         NaN  48.0

Then apply fillna and clean up:

res = res.assign(total=res['a'].fillna(res['b']).astype(int))\
         .reset_index().rename_axis('', 1)\
         .loc[:, ['name', 'total']]

print(res)

    name  total
0    one      7
1  three     39
2    two     48

Python: group by sum with condition

3 Answers3