1

I have a simple task to classify people by their height and hair length to either MAN or WOMAN category using a neural network. Also teach it the pattern with some examples and then use it to classify on its own.

I have a basic understanding of neural networks but would really need some help here.

I know that each neuron divides the area to two subareas, basically that is why P = w0 + w1*x1 + w2*x2 + ... + wn*xn is being used here (weights are just moving the line if we consider geometric representation).

I do understand that each epoche should modify the weights to get closer to correct result, yet I have never program it and I am hopeless about how to start.

How should I proceed, meaning: How can I determine the threshold and how should I deal with the inputs?

It is not a homework rather than task for the ones who were interested. I am and I would like to understand it.

joce
  • 9,624
  • 19
  • 56
  • 74
John V
  • 4,855
  • 15
  • 39
  • 63
  • I just prettified your question, but I'm unsure about the use of `epoche`. Do you mean [epoché](http://en.wikipedia.org/wiki/Epoch%C3%A9)? – joce Mar 30 '13 at 02:26

3 Answers3

3

Looks like you are dealing with a simple Perceptron with a threshold activation function. Have a look at this question. Since you ARE using a bias neuron (w0), you would set the threshold to 0.

You then simply take the output of your network and compare it to 0, so you would e.g. output class 1 if x < 0 and class 2 if x > 0. You could model the case x=0 as "indistinct".

For learning the weights you need to apply the Delta Learning Rule which can be implemented very easily. But be careful: a perceptron with a simple threshold activation function can only be correct if your data are linearly separable. If you have more complex data you will need a Multilayer Perceptron and a nonlinear activation function like the Logistic Sigmoid Function.

Have a look at Geoffrey Hintons Coursera Course, Lecture 2 for details.

Community
  • 1
  • 1
schreon
  • 1,097
  • 11
  • 25
1

I've been working with machine learning lately (but I'm not an expert) but you should look at the Accord.NET framework. It contains all the common machine learning algorithme out of the box. So it's easy to take an existing samples and modify it instead of starting from scratch. Also, the developper of the framework is very helpful in the forum available on the same page.

With the available samples, you may also discover something better than neural network like the Kernel Support Vector Machine. If you stick to the neural network, have fun modifying all the different variables and by tryout and error you will understand how it work.

Have fun!

Jean-François Côté
  • 4,200
  • 11
  • 52
  • 88
  • 1
    Excuse me, but it is not correct that SVMs are generally better than neural networks. Especially the latest developments regarding Deep Neural Networks outperform existing SVM based solutions. Take a look at the [MNIST records](http://yann.lecun.com/exdb/mnist/), look at the Convolution Net table and the SVM table. To state one example, there is no need to do preprocessing for DNNs, which is necessary for SVMs. Moreover you need to fine-tune SVMs as well: you have to choose the right kernel function and you have to set the parameters for your epsilon tube. – schreon Mar 26 '13 at 15:53
  • 1
    As I said, I'm not an expert, I was just trying to help. I suggested that you MAY discover something better than neural network, it all depends on the problem itself. In my own experience, the KSVM have always been easier to configure and took MUCH less ressources than the neural network. If you are interested in deep learning, the Accord.NET also has an example with that. – Jean-François Côté Mar 26 '13 at 17:10
0

Since you said:

I know that each neuron divides the area to two subareas

&

weights are just moving the line if we consider geometric representation

I think you want to use perseptron or ADALINE neural networks. These neural networks can just classify linear separable patterns. since your input data is complicated, It's better to use a Multi layer Non-Linear Neural network. (my suggestion is a two layer neural network with tanh activation function) . For training these network you should use back propagation algorithm.

For answering to

how should I deal with the inputs?

I need to know more details about the inputs( Like: are they just height and hair length or there is more, what is their range and your resolution and etc.)

If you're dealing with just height and hair length I suggest that divide heights and length in some classes (for example 160cm-165cm, 165cm-170cm & etc.) and for each one of these classes set an On/Off input neuron. then put a hidden layer after all classes related to heights and another hidden layer after all classes related to hair length (tanh activation function). Number of neurons in these two hidden layer is determined based on number of training cases. then take these two hidden layer output and send them to an aggregation layer with 1 output neuron.

mmohaveri
  • 528
  • 7
  • 23