2

I use Matlab's fminsearch function for finding the minimum with Nelder-Mead. fminsearch calculates the size of the initial simplex automatically. In my case, the initial simplex is too small, thus it performs not well.

fminsearch uses a leg of length 5% of the size of each variable, with a value of 0.00025 for zero variables. However, I have read the following (source):

However, it can be a reasonable policy to set the points in such a way that the simplex covers virtually the entire possible range. The algorithm of Nelder-Mead will shrink automatically the simplex and aproximate to the optimum. The practical advantage of this policy is that you will obtain a better overall-knowledge of the response-function.

What leg length (percentage) do I have to choose to admit to this policy?

Please remember that the first simplex-opereation is añways a reflection. If the starting simplex covers the whole permitted range the reflection necessarily will give a point off limits. But HillStormer allows to use linear constraints and can avoid this problem.

Which linear constraints do I have to use to avoid this problem? I'm using fminsearchbnd and fminsearchcon which allows such constraints.

Last but not least, I've read in Numerical Recipes that when the algorithm gets stuck at a local minima, it helps to reinitialize the simplex at the point where you get stuck. Of course, if fminsearch terminates I can rerun it with the new points as starting points. To what value should I set the initial simplex size in this case?

Community
  • 1
  • 1
machinery
  • 5,972
  • 12
  • 67
  • 118
  • It appears the `fminsearchbnd` function does the bounding for you. It seems to be quite a nice function for this problem. Just provide the boundaries to that function and if any parameter value goes beyond a boundary, it will be limited to the boundary. Your leg size can be set by making changes to the `fminsearch.m` file itself. I strongly recommend **not** changing the file itself, but making a copy for your needs. In the file, you can see lines containing `usual_delta = 0.05; ` and `zero_term_delta = 0.00025;`. They are a percentage of the variables. Try larger values and report back! – Erik May 24 '16 at 07:35
  • @Erik Thank you for your answer. If I set `usual_delta = 1`, does the initial simplex covers virtually the entire possible range? Why is the zero term delta so low? – machinery May 24 '16 at 10:00
  • 1 = 100%, which means the initial simplex will have legs of the size of the variables. The boundaries you wish to put on these variables can be different, e.g. if x is your variable and x0 = 1 the initial guess and you put the delta to 1, the simplex will begin with legs of size 1 around x, so 0 and 2 (correct me if I'm wrong). If your boundary is for example <=5, then your delta could be set larger. If the boundary you need were 1.2, then delta should be smaller. First determine your boundary, then determine x0, then the deltas you want. – Erik May 24 '16 at 10:15
  • I don't know why `zero_term_delta` is small. It is probably so small, because the user provides an initial guess of zero, so possibly the optimum can be found at zero. If the algorithm is allowed to take large steps away from zero it might find a local optimum instead of a global optimum and possibly computational cost increases too. – Erik May 24 '16 at 10:17
  • @Erik Thanks again. Is the percentage (for the legs) related to the initial guess of that variable and not the range? I assumed that it is related to the range, so that setting the delta to 1 (100%) means that the whole range is covered. – machinery May 24 '16 at 14:36
  • @Erik I know the boundaries, for example for parameter x it is [-5, 15] and for parameter y it is [0, 1]. The initial guesses x0 and y0 I determine by random sampling in the interval, so let's say x0 = -3 and y0 = 0.2. How can I calculate the delta for each variable so that it spans the whole range? Because I do random sampling for the initial guess, I need a somewhat automatic way. – machinery May 24 '16 at 14:41
  • You could try to make the delta very large, thus allowing the initial simplex to span the entire parameter space. I hope the bounded simplex function `fminsearchbnd` will simply force the legs to be no greater than the boundaries. However, then you'll never have random initialisation, because every initial simplex will be the same (spanning the entire parameter space) and the optimum found will always be the same too: it might not be the global optimum. – Erik May 25 '16 at 15:29
  • I see, I see...do you have another heuristic for choosing the delta? – machinery May 26 '16 at 09:09

0 Answers0