1

i'm trying to fill the empty y field in my dataset[from 12 to 16] , here is the original data vizualisation
enter image description here

I have NaN values at the end, here is my code

df=df.interpolate(method='polynomial', order=2, limit_direction ='both')

I have tried many methods (‘slinear’, ‘quadratic’, ‘cubic’, ‘spline’, ‘barycentric'), all of them gave the following result
enter image description here

As you see, from 12 to 14 the result is not the sinusoidal extension of my data, it just clone the first elements.

Moun
  • 325
  • 2
  • 16
  • 3
    Can you share your `df` or how you generated it? – Derek O Jun 12 '21 at 22:51
  • have you tried with a higher ```order``` (3 or 4) and/or ```limit_direction='forward'```? – 99_m4n Jun 12 '21 at 22:58
  • @DerekO I just generate the x data from 12 to 15 with x_extend=np.arange(start=np.max(x), stop=15, step=x_step) , step is the mean step of given dataset. then i concatenate the main dataset, with the generated one – Moun Jun 12 '21 at 23:05
  • @99_m4n yes I did, it gave the same result. – Moun Jun 12 '21 at 23:05
  • What about the given dataset? Can you provide a sample of it? – Derek O Jun 12 '21 at 23:10
  • 1
    @DerekO https://drive.google.com/file/d/1cBCqskU_Co-tc5EP3_9zMmHL5B9YGxmh/view?usp=sharing here is a link of the original one – Moun Jun 12 '21 at 23:11
  • Does this answer your question? [Extrapolate values in Pandas DataFrame](https://stackoverflow.com/questions/22491628/extrapolate-values-in-pandas-dataframe) – Derek O Jun 12 '21 at 23:55
  • @DerekO i tried what they did, the result wasn't there – Moun Jun 13 '21 at 11:01

1 Answers1

0

Based on the this answer, when you want to extrapolate data which is infer how the data behaves outside of the scope of interpolation, you need to rely on some more powerful interpolation functions from the scipy library or write an interpolation function on your own.

Although this may be a bit beyond what you were asking, I think your data is clearly a periodic function so polynomial functions or splines will continue unbounded past the endpoint of your original data set. Instead, trigonometric interpolation might extrapolate better due to its periodic behavior, and you can simplify the computational complexity by interpolating to equidistant points in your data set (e.g. don't fit a trigonometric function to all 500 points)

Edit: you can also use trigonometric least squares which is just least squares but with basis functions [1, cos(ax), sin(bx)]. I created an app that computes trigonometric least squares using a grid search to determine a,b. Feel free to repurpose the TrigPolynomial class I wrote in that app.

Derek O
  • 16,770
  • 4
  • 24
  • 43