-1

I was wondering which is the best machine learning technique to approximate a function that takes a 32-bit number and returns another 32-bit number, from a set of observations.

Thanks!

user1234299
  • 1,017
  • 1
  • 8
  • 14
  • 3
    could you elaborate a little on your purpose? there are tons of techniques, none of which can be considered as the best for one input and one output value. what are your observations, how many data points, is the data noisy, etc. – Regenschein Jun 25 '13 at 09:20
  • Sure. I'm using a black box technique to map certain values from a function f: (x1, y1), (x2, y2), .. (xn, yn) where xs and ys are 32-bits numbers, I should approximate or get x* for a certain y* that i know in advance such as f(x*) = *y. There is no noise and i can ask more for more data if is necesary.. – user1234299 Jun 25 '13 at 23:58

2 Answers2

1

Multilayer perceptron neural networks would be worth taking a look at. Though you'll need to process the inputs to a floating point number between 0 and 1, and then map the outputs back to the original range.

RutledgePaulV
  • 2,568
  • 3
  • 24
  • 47
  • Do you think it will be effective? – user1234299 Jun 25 '13 at 23:59
  • Yep. If there is a function, it should be able to identify it. If you look into the theory surrounding multilayer perceptrons, you'll come to find that it can learn any function if you use 3 hidden layers. As for how many neurons in each layer, you'll have to play around with it a little, usually slightly more than the number of inputs is all you need. – RutledgePaulV Jun 26 '13 at 00:48
  • Essentially, one hidden layer lets you separate into partially bounded spaces, two layers lets you define a single bounded space, and three layers allows you multiple bounded spaces. If I remember correctly.. ;) – RutledgePaulV Jun 26 '13 at 00:52
  • I agree with you, and with the theory, but i'm afraid that the transformation from 2^32 different states to a float in [0, 1] would be make really difficult to distinguish two points in the dataset because of the finite precision of floats.. – user1234299 Jun 26 '13 at 00:58
  • Well, the 0-1 depends on what activation function you use. I've found that you can use a standard sigmoid or tanh activation for the internal layers, and then use either an adjustable sigmoid or a linear activation function for the output layer. Then you can set the range yourself. I've had success doing that in the MLP I wrote. – RutledgePaulV Jun 26 '13 at 01:05
  • I understand your concern, but I think you'll find it's not really an issue - I'd at least look into it. Use doubles if all else fails..? Depending on the complexity of the function you won't need many neurons and it'll run pretty quickly if you use an efficient implementation. – RutledgePaulV Jun 26 '13 at 01:07
  • I will give it a try then. – user1234299 Jun 26 '13 at 01:09
0

There are several possible solutions to your problem:

1.) Fitting a linear hypothesis with least-squares method

In that case, you are approximating a hypothesis y = ax + b with the least squares method. This one is really easy to implement, but sometimes, a linear model is not good enough to fit your data. But - I would give this one a try first.

Good thing is that there is a closed form, so you can directly calculate parameters a and b from your data.

See Least Squares

2.) Fitting a non-linear model

Once seen that your linear model does not describe your function very well, you can try to fit higher polynomial models to your data.

Your hypothesis then might look like

y = ax² + bx + c

y = ax³ + bx² + cx + d

etc.

You can also use least squares method to fit your data, and techniques from the gradient descent types (simmulated annealing, ...). See also this thread: Fitting polynomials to data

Or, as in the other answer, try fitting a Neural Network - the good thing is that it will automatically learn the hypothesis, but it is not so easy to explain what the relation between input and output is. But in the end, a neural network is also a linear combination of nonlinear functions (like sigmoid or tanh functions).

Community
  • 1
  • 1
Regenschein
  • 1,514
  • 1
  • 13
  • 26