15

first, let me say that I lack experiences with scientific math or statistics - so this might be a very well-known problem, but I don't know where to start.

I have a function f(x1, x2, ..., xn) where I need to guess the x'ses and find the highest value for f. The function has the following properties:

  • the total number or parameters is usually around 40 to 60, so a brute-force approach is impossible.

  • the possible values for each x range from 0.01 to 2.99

  • the function is steady, meaning that a higher f value means that the guess for the parameters is better and vice versa.

So far, I implemented a pretty basic method in python. It initially sets all parameters to 1, randomly guesses new values and checks if the f is higher than before. If not, roll back to the previous values. In a loop with 10,000 iterations this seems to work somehow, but the result is propably far from being perfect.

Any suggestions on how to improve the search for the optimal parameters will be appreciated. When googling this issue things linke MCMC came up, but that seems like a very advanced method and I would need a lot of time to even understand the method. Basic hints or concepts would help me more than elaborated methods and algorithms.

Barmar
  • 741,623
  • 53
  • 500
  • 612
David
  • 153
  • 1
  • 1
  • 5
  • Welcome to SO! I'm sure there's a formal solution to this problem, but my first guess would be a greedy algorithm... For each `xn`, find the value of that `xn` that maximizes `f`, and repeat? This depends on the nature of `f` though. Thoughts? – A.Wan Jul 23 '13 at 15:16
  • Do you happen to know if the function has few maxima, or if it's expected to have a lot? If it's relatively few maxima, you could try [gradient descent](http://en.wikipedia.org/wiki/Gradient_descent) – Dan Lecocq Jul 23 '13 at 15:18
  • If you are sending an unknown number of variables to a function, I would suggest storing them in either an array, list, or dictionary. Probably a list for your case. – Josh Jul 23 '13 at 15:18
  • You could always use a list as one parameter unless your task says you cannot do that. A list would make it much easier. – Chase Burnham Jul 23 '13 at 15:21
  • Can we use derivatives here? – canbax Oct 09 '21 at 18:13

2 Answers2

11

Don't do it yourself. Install SciPy and use its optimization routines. scipy.optimize.minimize looks like a good fit.

user2357112
  • 260,549
  • 28
  • 431
  • 505
  • 1
    Thank you! I finally had the time to try that out, and it works like a charm. Now I need to fight the urge to have a look at the code in order to understand how it works :) – David Jul 24 '13 at 18:37
11

I think you want to take a look at scipy.optimize (http://docs.scipy.org/doc/scipy-0.10.0/reference/tutorial/optimize.html). A maximization is the minimization of the -1*function.

Jblasco
  • 3,827
  • 22
  • 25