4

I have read that Nelder Mead algorithm is working for unconstrained optimization. http://www.scholarpedia.org/article/Nelder-Mead_algorithm I think in Matlab Nelder Mead is used also for unconstrained optimization. However, I am a little bit confused, since I found a Java API for optimization http://www.ee.ucl.ac.uk/~mflanaga/java/Minimisation.html (Flanagan's Scientific Library) that has a class that implements Nelder Mead simplex and allows for defining constraints and bounds. So, is the version implemented in Flanagan's API a modified variation of the "classical" Nelder Mead algorithm?

Sorin
  • 105
  • 1
  • 10

1 Answers1

6

It looks like the API is implementing a simple "soft" constraint system, where constraints are transformed into penalty functions which severely penalize regions outside the constraints. It's a cheap-and-cheerful way of adding constraints to an unconstrained solver, but there'll be a tradeoff between optimality, convergence, and the degree to which the constraints are satisfied.

Sneftel
  • 40,271
  • 12
  • 71
  • 104
  • ok, and do you know is there any scientific article that describes this adaptation of the original algorithm? – Sorin Jan 03 '17 at 18:06
  • @Sorin I doubt there's one specifically for Nelder-Mead. Soft constraints are as old as dirt. Any good textbook on optimization will discuss them, though. – Sneftel Jan 03 '17 at 22:17
  • I was looking at the penalty weights proposed in the Flanagan's Scientific Library http://www.ee.ucl.ac.uk/~mflanaga/java/Minimisation.html#constraint but this appears to address only the individual simple constraints like x1>l or x1 – Sorin Jan 10 '17 at 15:30
  • @Sorin COBYLA https://docs.scipy.org/doc/scipy/reference/optimize.minimize-cobyla.html#optimize-minimize-cobyla is a similar simplex-based optimizer that allows for general nonlinear constraints. COBYLA is a significant improvement on Nelder-Mead. You can learn more: Powell, M J D. A direct search optimization method that models the objective and constraint functions by linear interpolation – MeowBlingBling Jan 22 '21 at 21:37