Python curve fit multiple parameters to multiple datasets

Question

I am trying to use scipy's curve_fit function to solve for model parameters. I used Python curve_fit with multiple independent variables as a starting point and was able to accomplish my needs, but now I'd like to use two input datasets to derive model parameters that would be shared by the two datasets (longer term I'd like to use many more than two data sets, but as a starting point I'm using two).

I thought the easiest way to do this might be to use curve_fit and input my data as a matrix. As a very contrived example I tried "augmenting" the example in the link above (I realize this isn't the prettiest code -- I'm just trying to get my bearings as to how I should actually be doing this).

def func(X, a, b, c):
    x,y = X
    result0 = np.log(a) + b*np.log(x[0]) + c*np.log(y[0])
    result1 = np.log(a) + b*np.log(x[1]) + c*np.log(y[1])
    return np.array([result0, result1])

# some artificially noisy data to fit
x0 = np.linspace(0.1,1.1,101)
y0 = np.linspace(1.,2., 101)

x1 = np.linspace(0.1,1.1,101)
y1 = np.linspace(1.,2., 101)

a, b, c = 10., 4., 6.

x = np.array([x0,x1])
y = np.array([y0,y1])

z = func((x,y), a, b, c)
z[0] = z[0] * 1 + np.random.random(101)/100
z[1] = z[1] * 1 + np.random.random(101)/100

# initial guesses for a,b,c:
p0 = 8., 2., 7.
print curve_fit(func, (x,y), z, p0)

This script returns the following error:

File "curveFitting.py", line 135, in print curve_fit(func, (x,y), z, p0) File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/optimize/minpack.py", line 533, in curve_fit res = leastsq(func, p0, args=args, full_output=1, **kw) File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/optimize/minpack.py", line 371, in leastsq raise TypeError('Improper input: N=%s must not exceed M=%s' % (n, m)) TypeError: Improper input: N=3 must not exceed M=2

Am I trying to use curve_fit in a way that is unintended/inappropriate? Is there another function/library I should be using instead?

score 3 · Accepted Answer · answered Jun 29 '18 at 20:16

The error is saying that you have 3 variables and 2 observations, which is not allowed: the number of variables must exceed the number of observations.

Your contrived example has 3 observations for each dataset -- this would be marginal, but with two such datasets it should work.

But, to make it work with curve_fit, your model function should use np.concatenate or np.flatten to make a one-dimensional array with the six observations for your 2 datasets of 3 observations each. That is, the value returned by the model function for curve_fit must be a 1-D array.

You ask about an alternative function or library: you might find lmfit useful. Among other features, it allows for multiple independent variables without the hack you link to. It would also work with your model function without having to use np.concatenate or flatten, as it will do this automatically for you. At this point, someone would comment if I did not state clearly that I am one of the authors of lmfit. Of course, I have no doubt that you would be able to see this from the lmfit docs and code.

Python curve fit multiple parameters to multiple datasets

1 Answers1