How to do a cubic or higher polynomial multiple regression in Python?

Question

I have a set of data where longitude and latitude are the independent variables and temperature is the dependent variable. I want to be able to perform extrapolation to get temperature values outside of the range of the latitude and longitude. The best way I thought to do this was to perform a multiple regression.

I know that sklearn has the functionality to perform a linear multiple regression from their linear_model library.

from sklearn import linear_model
regr = linear_model.LinearRegression()
regr.fit('independent data', 'dependent data') 

However, my temperature doesn't seem to have a linear relationship with the latitude or with the longitude. Thus, some of the values I extrapolate seem to be off.

I was thinking that I could perhaps improve the extrapolation by performing a polynomial multiple regression instead of a linear one.

Is there some library out there already that provides this functionality?

Answer 1

Probably the easiest way is to do linear regression but perform some basic 'feature engineering' and make your own polynomial features. You could take a look at PolynomialFeatures which can help construct an array of polynomial features.

As a basic example consider this:

# make example data
x = np.linspace(0, 10, 10)
y = x**2 + np.random.rand(len(x))*10

# make new polynomial feature
x_squared = x**2

# perform LR
LR = LinearRegression()
LR.fit(np.c_[x, x_squared], y) # np.c_ stacks the feature into a 2D array.

# evaulate the model
eval_x = np.linspace(0, 10, 100)
eval_x_squared = eval_x**2
y_pred = LR.predict(np.c_[eval_x, eval_x_squared])

# plot the result
plt.plot(x, y, 'ko')
plt.plot(eval_x, y_pred, 'r-', label='Polynomial fit')
plt.legend()

The resulting figure looks like this:

Of course we had to manually construct our features in this example, but hopefully it shows you how it can be practically implemented.