scipy.optimize get's trapped in local minima. What can I do?

Question

 from numpy import *; from scipy.optimize import *; from math import *
def f(X):
    x=X[0];    y=X[1]
    return x**4-3.5*x**3-2*x**2+12*x+y**2-2*y

bnds = ((1,5), (0, 2))
min_test = minimize(f,[1,0.1], bounds = bnds); 
print(min_test.x)

My function f(X)has a local minima at x=2.557, y=1 which I should be able to find.

The code showed above will only give result where x=1. I have tried with different tolerance and alle three method: L-BFGS-B, TNC and SLSQP. This is the thread I have been looking at so far: Scipy.optimize: how to restrict argument values

How can I fix this?

I am using Spyder(Python 3.6).

`print(f([2557, 1])) = 42690172880760.5`, I would not call this a local minimum... — Joe, Sep 21 '18 at 07:10
Try a [global optimization](https://en.wikipedia.org/wiki/Global_optimization) method instead. SciPy offers [some of them](https://docs.scipy.org/doc/scipy/reference/optimize.html#global-optimization). — toliveira, Sep 10 '21 at 02:14

Joe · Accepted Answer · 2019-09-16T17:30:14.557

You just encounterd the problem with local optimization: it strongly depends on the start (initial) values you pass in. If you supply [2, 1] it will find the correct minima.

Common solutions are:

use your optimization in a loop with random starting points inside your boundaries

import numpy as np
from numpy import *; from scipy.optimize import *; from math import *

def f(X):
    x=X[0];    y=X[1]
    return x**4-3.5*x**3-2*x**2+12*x+y**2-2*y

bnds = ((1,3), (0, 2))

for i in range(100):

    x_init = np.random.uniform(low=bnds[0][0], high=bnds[0][1])
    y_init = np.random.uniform(low=bnds[1][0], high=bnds[1][1])

    min_test = minimize(f,[x_init, y_init], bounds = bnds)

    print(min_test.x, min_test.fun)

use an algorithm that can break free of local minima, I can recommend scipy's basinhopping()
use a global optimization algorithm and use it's result as initial value for a local algorithm. Recommendations are NLopt's DIRECT or the MADS algorithms (e.g. NOMAD). There is also another one in scipy, shgo, that I have no tried yet.

Thank you. I have actually much problems with scipy.minimize when functions gets long and complicated. Can you recommend alternatives? — Kim, Sep 21 '18 at 08:09
What do you mean with long and complicated? There are some other optimization modules, e.g. `nlopt`. But they all will use your long and complicated function. In the end the methods available in scipy are fine for a large variety of problems. — Joe, Sep 21 '18 at 08:24

score 6 · Answer 2 · answered Sep 17 '19 at 07:52

Try scipy.optimize.basinhopping. It simply just repeat your minimize procedure multiple times and get multiple local minimums. The minimal one is the global minimum.

minimizer_kwargs = {"method": "L-BFGS-B"}
res=optimize.basinhopping(nethedge,guess,niter=100,minimizer_kwargs=minimizer_kwargs)

scipy.optimize get's trapped in local minima. What can I do?

2 Answers2

Linked