How to set variable arguments in scipy.optimize.curve_fit in Python?

Question

I am using scipy.optimize.curve_fit to do an optimization problem in Python. The curve_fit function takes inputs as curve_fit(f, xdata, ydata, ...), in which f callable is the model function, f(x, ...). It must take the independent variable as the first argument and the parameters to fit as separate remaining arguments.

However, in my case, the remaining arguments are variable. For example, f can be f = fun(indepent_var, a, b, c, d), or f = fun(indepent_var, a, c, d).

The actual arguments of f are defined by users when solving the optimization problem. For example, one user might want to use a, b, c, d as the arguments, and another user wants to use a, c, d, and the 3rd user might use b, c. Then, my question is how to set variable arguments in f and then users can configurize their own arguments?

def func(data, a, b, c, d):
    return a * np.exp(-b * data) + c        

#def func(data, a, c, d):
#    return a * np.exp(-c * x)

popt, pcov = curve_fit(func, xdata, ydata)

"config_1.ini"
params_to_be_optimized = a, b, c, d

"config_2.ini"
params_to_be_optimized = a, c, d

I am trying to do something like this:

def func(data, **kwargs):
    a = kwargs['a'] if a in kwargs else 0
    ...
    return ...
    
popt, pcov = curve_fit(lambda(...), xdata, ydata)

But I don't know exactly how to solve the problem. I suppose lambda function may work, but can anyone give me an example?

Answer 1

With the example in the curve_fit docs:

In [260]: def func(x, a, b, c):
     ...:     return a * np.exp(-b * x) + c
     ...: 
In [261]: xdata = np.linspace(0, 4, 50)
     ...: y = func(xdata, 2.5, 1.3, 0.5)
     ...: np.random.seed(1729)
     ...: y_noise = 0.2 * np.random.normal(size=xdata.size)
     ...: ydata = y + y_noise

Without a p0 parameter the number of fit parameters is determined by introspection, that is, examining the func code and attributes

p0 : None, scalar, or N-length sequence, optional

Initial guess for the parameters. If None, then the initial values will all be 1 (if the number of parameters for the function can be determined using introspection, otherwise a ValueError is raised).

In this case it identifies 3 parameters:

In [263]: optimize.curve_fit(func, xdata, ydata)
Out[263]: 
(array([ 2.55423706,  1.35190947,  0.47450618]),
 array([[ 0.0158905 ,  0.00681778, -0.0007614 ],
        [ 0.00681778,  0.02019919,  0.00541905],
        [-0.0007614 ,  0.00541905,  0.00282595]]))

I can define a equivalent function using *args:

In [264]: def f1(x, *args):
     ...:     return func(x, *args)
     ...: 

But if I try to fit it, I get an error:

In [266]: optimize.curve_fit(f1, xdata, ydata)
ValueError: Unable to determine number of fit parameters.

But I can specify a p0:

In [268]: optimize.curve_fit(f1, xdata, ydata, p0=np.ones(3))
Out[268]: 
(array([ 2.55423706,  1.35190947,  0.47450618]),
 ....)

If I give other p0 sizes I get errors as f1 passes its arguments on to func. I could define f1 so it was more tolerant of other args numbers, but I might then get OptimizeWarning.

Answer 2

This may not be a complete answer, but is too long for a comment. For sure, providing a more concrete example of what you're trying to do would definitely help clarify your question and lead to better answers.

It's not entirely clear to me what the signature of your function is, but it may be that the basic question is how to have the user select which of the function arguments become variables in the fit while having other arguments not become variables, either remaining at some fixed value or being used as switches for what functional form to actually use (which seems like what your clarifying comment was trying to say).

In either case, you may find lmfit (https://lmfit.github.io/lmfit-py/) useful. It has a Model class that supports curve-fitting based on scipy.optimize but separate from (and somewhat higher level than) curve_fit. lmfit.Model can turn any "model function" into a Model that can be used to fit to data, and uses inspection to turn the function arguments into Parameters used in the fit. Any of the Parameters can be variable or fixed (or have bounds or be set by a simple constraint expression), which might be what you're looking for: the user can decide that some of the function arguments would not be fit variables.

Also, if the model function has keyword arguments that don't have numerical defaults, they do not become fit variables, but can be used as optional switches, perhaps controlling the functional form used. That is, if you have a model function like

def func(x, a=1, b=2, c=3, d=0, option=True):
     if option:
         return a * np.exp(-b*x) + c + d*x
     else:
         return a * np.exp(-b*x**2) + c + d*x

Then you could create a fitting model and parameters with

from lmfit impor Model
mymodel = Model(func)
params = mymodel.make_params(a=100, b=-1, c=0, d=0)

and then (for example) fix d to be 0:

params['d'].value = 0
params['d'].vary = False

or apply other starting values, bounds, etc.

To change the value of the option argument, you would treat it as another independent variable, passing it to mymodel.fit(), or mymodel.eval() as a keyword argument, like this:

xdata = <array of data>
ydata = <array of data>

result = mymodel.fit(ydata, params, x=xdata, option=False)
print(result.fit_report())

That would allow the user to select the options used in the model function.

Hope that points you in the right direction....

After updates to question:

In general, supporting **kws would be very hard: the user can pass in anything to this dict and how it is used in the function is unknown. Fitting variables have to be floating point numbers, for example. But, you might also recognize that

def func(data, **kwargs):
    a = kwargs['a'] if a in kwargs else 0
    ...

can be restated as

def func(data, a=0, **kwargs):
    ...

and this is supported. That would mean you would have to explicitly make function arguments for all quantities that could potentially be variables, but that should not be too hard. Once that is done, the user can decide which of these are actually varied in a particular fit.

You can definitely use something like

def func(x, a=1, b=2, c=3, d=0):
     ...

# read from "config_1.ini"
params_to_be_optimized = 'a', 'b', 'c', 'd'

# read from "config_2.ini"
params_to_be_optimized = 'a', 'c', 'd'

and something like

mymodel = Model(func)
params = mymodel.make_params()   

for parname, param in params.items():
    param.vary = parname in params_to_be_optimized

to control which parameters will be optimized and which will be fixed.

Answer 3

really, you can only refer to curve_fit Signature ONLY - and in lib's module minpack.py is definitely told

"if 'args' in kwargs: The specification for the model function f does not support additional arguments. Refer to the curve_fit docstring for acceptable call signatures of f."

• again refering to Signature of the curve_fit function (where input of additional non-optimized parameters is absent) –

the only programming Trick you can use:

1. define your arg outside the function & before curve_fit function to use it in optimization process inside the objective (for the computer its variable value will be already know) -- # IF variable arg is being defined in the module, where curve_fit & objective functions exist
2. make arg variable Global - # IF variable arg is being defined in another module from where curve_fit & objective functions exist
3. or better import that module as usually & use var from it without need to define it as global

p.s. global vars is not good, though can be used if needed... too large topic for discussion - no sence here

it's just a Trick

It's all just about Code_Design