How does the SLSQP optimization algorithm work?

Question

I am using the SLSQP algorithm in openMDAO, but I am having trouble understanding how it actually works. I am just looking at the common paraboloid example, which has 2 design variables and aims to minimise f, without any constraints. By printing out the values of x,y and f for each iteration (iteration is probably not the right word for this), I can see that occasionally the first derivative is evaluated using forward finite difference for each design variable(x, y). These derivatives are then used to find the next x and y values, however I cannot see the pattern.

Also, when I read about the SLSQP method, the second derivatives are also required. However, I do not see it being calculated. Let me give an example of my output:

iteration 1:
x = 0
y = 0
f = 22

iteration 2:
x = 0.01
y = 0
f = 21.9401

iteration 3:
x = 0
y = 0.01
f = 22.0801

from these last 2 iterations we can calculate, df/dx = 5.99 , df/dy = -8.01

The next iteration happens to be:

x = 5.99
y = -8.01
f = -25.9597

Then again two finite difference calculations from this point to find: df/dx = 2.02 , df/dy = 2.02

Then the next iteration has variables: x = 8.372726, y = -6.66007 And I have no idea how to obtain those values.

Also, sometimes a large step is taken without even calculating the derivatives at that point. Possibly because the previous step was too large resulting in the function going away from the minimum.

I am hoping someone can explain to me or give a useful source for the exact algorithm that is used, or give any tips that could be used to better understand it. Thanks a lot!

The exact algorithm is explained in the paper `Kraft, Dieter. "A software package for sequential quadratic programming." Forschungsbericht- Deutsche Forschungs- und Versuchsanstalt fur Luft- und Raumfahrt (1988)`., but it might be hard to obtain. For a more general treatment, `Nocedal, Jorge, and Stephen J. Wright. "Sequential quadratic programming." Numerical optimization (2006): 529-562.` should help. The code itself comes from [toms-733](http://www.netlib.org/toms/733), which has an additional [source](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.512.2567&rep=rep1&type=pdf) — sascha, Jan 19 '20 at 11:11
The question is good, but would'nt the https://math.stackexchange.com/ community be a more appropriate place? — Michael Baudin, Oct 05 '20 at 07:43

score 3 · Answer 1 · edited Jan 07 '22 at 16:53

The algorithm described by Dieter Kraft is a quasi-Newton method (using BFGS) applied to a Lagrange function consisting of loss function and equality- and inequality constraints. Because at each iteration some of the inequality constraints are active, some not, the inactive inequalities are omitted for the next iteration. An equality constrained problem is solved at each step using the active subset of constraints in the Lagrange function.

How does the SLSQP optimization algorithm work?

1 Answers1