On fitting a curved line to a dataset

Question

I have a plot with two data sets which produces a slight gradient, where a curved line of best fit may be overplotted.

At the moment I have only managed to get a straight line of best fit. I understand scipy.optimize.curve_fit should be able to help me, but this requires me to know the function I want to overplot (I think).

Below are my code and plots. How would one go about creating a curved plot for these data sets?

plt.figure(figsize=(15,6.6))
pl.subplot(1,2,1) 
plt.plot(gg,AA, 'kx')
plt.xlabel('x')
plt.ylabel('y')
plt.gca().invert_yaxis()
y=AA
x=gg
fit=pl.polyfit(x,y,1)
#slope, fit_fn=pl.poly1d(fit)
fit_fn=pl.poly1d(fit)
scat=pl.plot(x,y, 'kx', x,fit_fn(x), '-b' )


pl.subplot(1,2,2) 
pl.plot(LL,pp, 'kx')#shows points with no removal or bestfit
plt.gca().invert_yaxis()

plt.savefig('1.jpg')
plt.show()

It should be noted that there is possibly no curve but I want to discover if there is one which would fit.

Answer 1

If I understand well, your question is much rather a conceptual than a practical one.

If you want to show a line that somewhat represents your dataset, you could start with three things: moving average, interpolation and polynomial fit.

Moving average smoothes your dataset nicely. I'm not aware of a built-in function for it, but you can code it yourself, as it was discussed here.

Interpolation (spline, for example) fits some function on your dataset which can be evaluated at many points and then plotted.

With the two mentioned methods, you don't have to specify a function. However, you can fit a polynomial function yourself. How to determine the degree of the polynomial? You can take the log of all your data points, fit a linear line to the log data, and IF IT FITS WELL, the coefficient of the linear part can be considered as the degree of the polynomial to the original dataset. However, don't use too large degree of polynomials - you can easily run into overfitting with this method.