46

I have inherited some code that is trying to minimize a function using scipy.optimize.minimize. I am having trouble understanding some of the inputs to the fun and jac arguments.

The call to minimize looks something like this:

result = minimize(func, jac=jac_func, args=(D_neg, D, C), method = 'TNC' ...other arguments)

func looks like the following:

def func(G, D_neg, D, C):
    #do stuff

jac_func has the following structure:

def jac_func(G, D_neg, D, C):
    #do stuff

What I don't understand is where the G input to func and jac_func is coming from. Is that somehow specified in the minimize function, or by the fact that the method is specified as TNC? I've tried to do some research into the structure of this optimization function but I'm having trouble finding the answer I need.

cottontail
  • 10,268
  • 18
  • 50
  • 51
sedavidw
  • 11,116
  • 13
  • 61
  • 95

2 Answers2

65

The short answer is that G is maintained by the optimizer as part of the minimization process, while the (D_neg, D, and C) arguments are passed in as-is from the args tuple.

By default, scipy.optimize.minimize takes a function fun(x) that accepts one argument x (which might be an array or the like) and returns a scalar. scipy.optimize.minimize then finds an argument value xp such that fun(xp) is less than fun(x) for other values of x. The optimizer is responsible for creating values of x and passing them to fun for evaluation.

But what if you happen to have a function fun(x, y) that has some additional parameter y that needs to be passed in separately (but is considered a constant for the purposes of the optimization)? This is what the args tuple is for. The documentation tries to explain how the args tuple is used, but it can be a little hard to parse:

args: tuple, optional

Extra arguments passed to the objective function and its derivatives (Jacobian, Hessian).

Effectively, scipy.optimize.minimize will pass whatever is in args as the remainder of the arguments to fun, using the asterisk arguments notation: the function is then called as fun(x, *args) during optimization. The x portion is passed in by the optimizer, and the args tuple is given as the remaining arguments.

So, in your code, the value of the G element is maintained by the optimizer while evaluating possible values of G, and the (D_neg, D, C) tuple is passed in as-is.

lmjohns3
  • 7,422
  • 5
  • 36
  • 56
  • 3
    Nice explanation! Usually you would give a guess as to what you expect `G` to be, say `G0`. So instead of the form in the OP, you'd use `minimize(func, jac=jac_func, G0, args=(...,), ...)`. If you don't pass one, it usually starts at `G=1.` I think it's less confusing in this form, even if `G0=1.` as well. – askewchan Nov 07 '13 at 20:20
  • 1
    I agree, it'd be nice to be more explicit about the initial guess for `G0` -- in fact I'd never seen a call to `minimize` that omitted it. – lmjohns3 Nov 07 '13 at 21:27
  • 5
    @lmjohns3 @askewchan: the initial guess `x0` is not an optional parameter, as you can see in the documentation. The user must always explicitly specify it, and there is no default value. – pv. Nov 07 '13 at 22:46
  • @pv. sure enough, thanks ! I've updated my answer just a bit to make it sound less like the minimizer is creating values for `x` from nowhere. – lmjohns3 Nov 07 '13 at 22:56
  • @pv. good catch --- as it happens almost all of the `scipy.optimize` functions require the starting point, except of course the one I used most recently: `curve_fit`, which indeed has a default of `1.0`. I wonder why it's missing from the OP. – askewchan Nov 08 '13 at 01:36
0

For illustration purposes, we can print how G changes as minimize iterates to the local minimum. For example, consider the following case where the objective is to minimize a quadratic function of the form f(x) = 2*x**2 - 4*x + 7 (which attains its minimum at x=1). Note that the parameters of the function (2, -4 and 7) were supplied as arguments to obj_func below.

The initial guess has to be supplied to start the algorithm.

As the output shows, starting from the initial value of 10, the function variable G descends to the minimizer.

from scipy.optimize import minimize

def obj_func(G, a, b, c):
    print(G)
    return a*G**2 + b*G + c

initial_guess = 10
a, b, c = 2, -4, 7

result = minimize(obj_func, x0=initial_guess, args=(a, b, c))

print(f"\nMinimizer = {result.x}")
print(f"  Minimum = {result.fun}")

which outputs the following:

[10.]
[10.00000001]
[8.99]
[8.99000001]
[6.82085113]
[6.82085114]
[1.62275043]
[1.62275045]
[1.]
[1.00000001]

Minimizer = [1.]
  Minimum = 5.0

Another example: Consider a two-variable function of the form f(x, y) = (x - 1)**2 + (y - 2)**2. The function attains its minimum at (x, y) = (1, 2) (and the minimum is 0).

Then starting from an initial point of (x, y) = (0, 3), the function converges to the minimum as in the following.

def obj_func(variable, x_offset, y_offset):
    x, y = variable
    print(f"x={x:.3f}, y={y:.3f}")
    return (x - x_offset)**2 + (y - y_offset)**2

initial_guess = [0, 3]
result = minimize(obj_func, initial_guess, args=(1, 2))

This prints the following which shows that the variables converge to the minimizer.

x=0.000, y=3.000
x=0.000, y=3.000
x=0.000, y=3.000
x=0.714, y=2.286
x=0.714, y=2.286
x=0.714, y=2.286
x=1.000, y=2.000
x=1.000, y=2.000
x=1.000, y=2.000

An important note about minimize is that the initial guess has to be an educated one especially if the objective function is complex, otherwise the algorithm may not converge. I find that to be a major source of unsuccessful optimization run.

cottontail
  • 10,268
  • 18
  • 50
  • 51