2

I'm using the fminsearch Method of Matlab to minimize a function:

c = cvpartition(200,'KFold',10);
minfn = @(z)kfoldLoss(fitcsvm(cdata,grp,'CVPartition',c,...
    'KernelFunction','rbf','BoxConstraint',exp(z(2)),...
    'KernelScale',exp(z(1))));
opts = optimset('TolX',5e-4,'TolFun',5e-4);
[searchmin fval] = fminsearch(minfn,randn(2,1),opts)

The minimization is over two parameters.

Now I would like to minimize a third parameter, but this parameter can only take positive integer values, i.e. 1,2,3,...

How can I tell fminsearch to only consider positive integers?

Second, if my third parameter gets initialized to 10 but it actual best value is 100, does fminsearch converge fast in such cases?

machinery
  • 5,972
  • 12
  • 67
  • 118

3 Answers3

3

You can't tell fminsearch to consider only integers. The algorithm it uses is not suitable for discrete optimization, which in general is much harder than continuous optimization.

If there are only relatively few plausible values for your integer parameter(s), you could just loop over them all, but that might be too expensive. Or you could cook up your own 1-dimensional discrete optimization function and have it call fminsearch for each value of the integer parameter it tries. (E.g., you could imitate some standard 1-dimensional continuous optimization algorithm, and just return once you've found a parameter value that's, say, better than both its neighbours.) You may well be able to adapt this function to the particular problem you're trying to solve.

Gareth McCaughan
  • 19,888
  • 1
  • 41
  • 62
  • Separating the integer from the continuous optimization problem seems great. Do you know other 1D or nD discrete optimization procedures? – machinery May 10 '16 at 12:54
  • There are lots. Whether any will suit your particular needs, I don't know. You could start with, say, the [Wikipedia page](https://en.wikipedia.org/wiki/Discrete_optimization) on discrete optimization. – Gareth McCaughan May 10 '16 at 13:12
1

As @Gareth McCaughan said, you can't tell fminsearch to restrict the search space to integers. If you want to search for solvers that can handle this type of problem, you want to search for "mixed integer programming." Mixed integer is for part continuous, part integer programming. And "programming" is jargon for optimization (horribly confusing name, but like the QWERTY keyboard, we're stuck with it).

Be aware though that integer programming is in general NP-hard! Larger problems may be entirely intractable.

Matthew Gunn
  • 4,451
  • 1
  • 12
  • 30
1

In side the case I handled, i looked for an vector-index which satifies a condition. The vector-Index is postive integer. The workaround for fminsearch I did, was an interpolation of the error-function. Assume, fminsearch proposes 5.1267 as new index. Than I calculated the error-function for indexes 5 and 6 and gave an interpolation back. This leaded to stable and satisfying results.

Holger.Lindow@plr-magdeburg.de

user7762508
  • 11
  • 1