How to get the percent change of values in a dataframe while caring about NaN values?

Question

I have the following DataFrame:

   Date                 A                
   2015-01-01          10               
   2015-01-02          14               
   2015-01-05          NaN               
   2015-01-06          NaN                
   2015-01-07          5
   2015-01-10          1  
   2015-01-11          NaN  
   2015-01-12          21  
   2015-01-14          13

And I would like to get a data frame with the pct change in the data frame only if this two values are adjacent and not separated by NaN.

i.e. I would like this:

        Date                 A                             
       2015-01-02         0.2857                           
       2015-01-10         -0.8  
       2015-01-14         -0.38

But if I do: df.pct_change() it will return the pct change between 14 and 5 for the 201-01-07

@ayhan, this is perfect! `df.assign(A=df.A.pct_change(fill_method=None)).dropna()` I think you should post it as an answer! — MaxU - stand with Ukraine, Apr 20 '17 at 19:18
@MaxU Thanks. Looks like it is solved already so let me leave it as a comment (I didn't know if Date was an index, if this was a Series etc that's why I didn't answer). — ayhan, Apr 20 '17 at 20:42

score 5 · Accepted Answer · answered Apr 20 '17 at 18:58

5

Use pct_change and shift:

df.pct_change()[df.shift(1).notnull()].dropna()

Output:

                   A
Date                
2015-01-02  0.400000
2015-01-10 -0.800000
2015-01-14 -0.380952

answered Apr 20 '17 at 18:58

Scott Boston

147,308
15
139
187

score 3 · Answer 2 · answered Apr 20 '17 at 18:54

3

use cumsum on isnull to find groups to groupby

s = df.set_index('Date').A
c = n.cumsum()
s.groupby(c).pct_change().dropna().reset_index()

        Date         A
0 2015-01-02  0.400000
1 2015-01-10 -0.800000
2 2015-01-14 -0.380952

answered Apr 20 '17 at 18:54

piRSquared

285,575
57
475
624

How to get the percent change of values in a dataframe while caring about NaN values?

2 Answers2

Linked