0

I have 3 independent variables that I want to fit to a form:

y= a + b*x1 + c*x2 + d*x3

where I can arbitrarily impose constraints on a, b, c, and d.

I somewhat understand how to do this in cvxpy but my linear algebra education was 10 years ago and I can't seem to find an example on SO or other places on the web that include an intercept in a multilinear regression...

The code in this previous SO answer is almost what I want except doesn't seem to support the intercept...: https://stackoverflow.com/a/39853507/13128034

from cvxpy import *

a = np.array([1.2, 2.3, 4.2])
b = np.array([1, 5, 6])
c = np.array([5.4, 6.2, 1.9])

m = np.vstack([a,b,c])
y = np.array([5.3, 0.9, 5.6])

X = Variable(m.shape[0])
constraints = [X >= 0, sum_entries(X) == 1.0]

product = m.T * diag(X)
diff = sum_entries(product, axis=1) - y
problem = Problem(Minimize(norm(diff)), constraints)
problem.solve(verbose=True)

print(problem.value)
print(X.value)

I assume I just need to modify the m and X arrays in this example to include an intercept?

pythonlearning
  • 1
  • 1
  • What kind of constraints do you want to impose? – swag2198 Apr 25 '21 at 14:43
  • @swag2198 generally just a/b/c/d > 0 or = 0. I mostly understand how to do that in cvxpy though from examples on SO and in the documentation. – pythonlearning Apr 25 '21 at 15:28
  • To account for the intercept, you can simply append an extra `1` to the regression variable. You simply need to add a constraint that `X[n] == 1` to the variable which will give `w1x1 + w2x2 + ... + wn` after multiplying with the `m` matrix, where `wn` is the intercept. – swag2198 Apr 25 '21 at 17:17
  • @swag2198 if I do X=Variable(m.shape[0]+1) I get an incompatible dimensions error (3,3)(4,4) on the next line (m.T * diag(X)) which seems to imply that I need to have another column in m? – pythonlearning Apr 25 '21 at 20:41
  • Definitely. The example does not use the intercept parameter! – swag2198 Apr 25 '21 at 22:32
  • so this is where my inability to remember my lin.alg. is coming in....do I add a column of 1's? 0's? the other columns are independent variables but by definition the intercept doesnt have a variable...right? – pythonlearning Apr 25 '21 at 22:58
  • No no, the column comes from the data you have. Basically you have these data for the coefficients and intercepts and the corresponding regression output i.e sets of (y, a, b, c, d) following your notation. Better watch some tutorial on basic linear regression and matrix calculations if you are not clear. – swag2198 Apr 25 '21 at 23:16

0 Answers0