How using python to groupby and scaling values?

Question

I would like to rescaled column 'w'.

I have averaged 'w'.

aveData_set = Data_Set.groupby(['buildingid', pd.Grouper(key='reporttime',freq='15T')])['w'].mean().reset_index()

aveData_set result:

aveData_set result

Then I would like each 24H rescaling column 'w'.

ScaleData_set = aveData_set.groupby(['buildingid', pd.Grouper(key='reporttime',freq='24H')])['w'].apply(lambda x: (x-x.min())/(x.max()-x.min())).reset_index()

But result is strange,some column have disappeared.

ScaleData_set result:

ScaleData_set result

I really need your help.Many thanks.

jezrael · Accepted Answer · 2018-06-09T06:40:10.783

Output is expected, because scaling not aggregate values. It return Series with same size as original DataFrame.

So is possible create new column:

aveData_set['w_scaled'] = (aveData_set.groupby(['buildingid',
                                                pd.Grouper(key='reporttime',freq='1d')])['w']
                                      .apply(lambda x: (x-x.min())/(x.max()-x.min())))

Or reassign back:

aveData_set['w'] = (aveData_set.groupby(['buildingid',
                                          pd.Grouper(key='reporttime',freq='1d')])['w']
                               .apply(lambda x: (x-x.min())/(x.max()-x.min())))

Here apply working like transform, check better explanation here with similar lambda function:

aveData_set['w'] = (aveData_set.groupby(['buildingid',
                                          pd.Grouper(key='reporttime',freq='1d')])['w']
                               .transform(lambda x: (x-x.min())/(x.max()-x.min())))

Sample:

rng = pd.date_range('2017-04-03 18:09:04', periods=10, freq='7T')
Data_Set = pd.DataFrame({'reporttime': rng, 'w': range(10), 'buildingid':[39] * 5 + [40] * 5})
print (Data_Set)
           reporttime  w  buildingid
0 2017-04-03 18:09:04  0          39
1 2017-04-03 18:16:04  1          39
2 2017-04-03 18:23:04  2          39
3 2017-04-03 18:30:04  3          39
4 2017-04-03 18:37:04  4          39
5 2017-04-03 18:44:04  5          40
6 2017-04-03 18:51:04  6          40
7 2017-04-03 18:58:04  7          40
8 2017-04-03 19:05:04  8          40
9 2017-04-03 19:12:04  9          40

aveData_set = (Data_Set.groupby(['buildingid', 
                                 pd.Grouper(key='reporttime',freq='15T')])['w']
                       .mean().reset_index())
print (aveData_set)

   buildingid          reporttime    w
0          39 2017-04-03 18:00:00  0.0
1          39 2017-04-03 18:15:00  1.5
2          39 2017-04-03 18:30:00  3.5
3          40 2017-04-03 18:30:00  5.0
4          40 2017-04-03 18:45:00  6.5
5          40 2017-04-03 19:00:00  8.5

aveData_set['w'] = (aveData_set.groupby(['buildingid',
                                          pd.Grouper(key='reporttime',freq='1d')])['w']
                               .apply(lambda x: (x-x.min())/(x.max()-x.min())))

print (aveData_set)
   buildingid          reporttime         w
0          39 2017-04-03 18:00:00  0.000000
1          39 2017-04-03 18:15:00  0.428571
2          39 2017-04-03 18:30:00  1.000000
3          40 2017-04-03 18:30:00  0.000000
4          40 2017-04-03 18:45:00  0.428571
5          40 2017-04-03 19:00:00  1.000000

I am a python newbie and thank you for your help at one time. — Linminxiang, Jun 09 '18 at 06:36

How using python to groupby and scaling values?

1 Answers1