Single prediction with linear regression

Question

Implementing linear regression as below:

from sklearn.linear_model import LinearRegression

x = [1,2,3,4,5,6,7]
y = [1,2,1,3,2.5,2,5]

# Create linear regression object
regr = LinearRegression()

# Train the model using the training sets
regr.fit([x], [y])

# print(x)
regr.predict([[1, 2000, 3, 4, 5, 26, 7]])

produces :

array([[1. , 2. , 1. , 3. , 2.5, 2. , 5. ]])

In utilizing the predict function why cannot utilize a single x value in order to make prediction?

Trying regr.predict([[2000]])

returns:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-3a8b477f5103> in <module>()
     11 
     12 # print(x)
---> 13 regr.predict([[2000]])

/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/base.py in predict(self, X)
    254             Returns predicted values.
    255         """
--> 256         return self._decision_function(X)
    257 
    258     _preprocess_data = staticmethod(_preprocess_data)

/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/base.py in _decision_function(self, X)
    239         X = check_array(X, accept_sparse=['csr', 'csc', 'coo'])
    240         return safe_sparse_dot(X, self.coef_.T,
--> 241                                dense_output=True) + self.intercept_
    242 
    243     def predict(self, X):

/usr/local/lib/python3.6/dist-packages/sklearn/utils/extmath.py in safe_sparse_dot(a, b, dense_output)
    138         return ret
    139     else:
--> 140         return np.dot(a, b)
    141 
    142 

ValueError: shapes (1,1) and (7,7) not aligned: 1 (dim 1) != 7 (dim 0)

Answer 1

When you do this:

regr.fit([x], [y])

You are essentially inputing this:

regr.fit([[1,2,3,4,5,6,7]], [[1,2,1,3,2.5,2,5]])

that has a shape of (1,7) for X and (1,7) for y.

Now looking at the documentation of fit():

Parameters:

X : numpy array or sparse matrix of shape [n_samples,n_features]
    Training data

y : numpy array of shape [n_samples, n_targets]
    Target values. Will be cast to X’s dtype if necessary

So here, what the model assumes it that you have data which have data has 7 features and you have 7 targets. Please see this for more information on multi-output regression.

So at the prediction time, model will require data with 7 features, something of shape (n_samples_to_predict, 7) and will output the data with shape (n_samples_to_predict, 7).

If instead, you wanted to have something like this:

  x   y
  1  1.0
  2  2.0
  3  1.0
  4  3.0
  5  2.5
  6  2.0
  7  5.0

then you need to have a shape of (7,1) for input x and (7,) or (7,1) for target y.

So as @WStokvis said in comments, you need to do this:

import numpy as np
X = np.array(x).reshape(-1, 1)
y = np.array(y)          # You may omit this step if you want

regr.fit(X, y)           # Dont wrap it in []

And then again at prediction time:

X_new = np.array([1, 2000, 3, 4, 5, 26, 7]).reshape(-1, 1)
regr.predict(X_new)

And then doing the following will not raise error:

regr.predict([[2000]])

because the required shape is present.

Update for the comment:-

When you do [[2000]], it will be internally converted to np.array([[2000]]), so it has the shape (1,1). This is similar to (n_samples, n_features), where n_features = 1. This is correct for the model because at the training, the data has shape (n_samples, 1). So this works.

Now lets say, you have:

X_new = [1, 2000, 3, 4, 5, 26, 7] #(You havent wrapped it in numpy array and reshape(-1,1) yet

Again, it will be internally transformed as this:

X_new = np.array([1, 2000, 3, 4, 5, 26, 7])

So now X_new has a shape of (7,). See its only a one dimensional array. It doesn't matter if its a row vector or a column vector. Its just one-dimensional array of (n,).

So scikit may not infer whether its n_samples=n and n_features=1 or other way around (n_samples=1 and n_features=n). Please see my other answer which explains about this.

So we need to explicitly convert the one-dimensional array to 2-d by reshape(-1,1). Hope its clear now.