0

Are there any general rules for how many layers/neurons are needed to solve a given problem?

I understand that 1 hidden layer is sufficient for patterns that are linearly separable (such as the AND logic gate), and 2 hidden layers for patterns that are not (XOR gate). I would like to make one that can learn to separate 3 types of object given 2 parameters.

I have attached an image for clarity.

three types of objects

Giewev
  • 174
  • 2
  • 10
user5351703
  • 31
  • 1
  • 3
  • 1
    You'll have to evaluate empirically how many layers give you the best performance. – Lars Kotthoff Oct 23 '15 at 02:18
  • 1
    Possible duplicate of [multi-layer perceptron (MLP) architecture: criteria for choosing number of hidden layers and size of the hidden layer?](http://stackoverflow.com/questions/10565868/multi-layer-perceptron-mlp-architecture-criteria-for-choosing-number-of-hidde) – Giewev Oct 23 '15 at 16:46

1 Answers1

2

I understand the use of 1 hidden layer for patterns that are linearly separable (such as the AND logic gate) and 2 hidden layers for patters that are not (XOR gate)

That's not true: one hidden layer is also often enough for patterns that are not linearly separable.

The number of neurons in the layers is what matters more for theoretical separability. A single layer with 3-4 neurons will do very well very fast on the XOR problem.

As for how to choose the actual number, for some problems it's well known what works best. There are also various heuristics (for example, for a single hidden layer, have sqrt(input layer size + output layer size) is a basic one).

Generally, you have to try multiple options, probably using a grid search, and see what works better for your problem.

IVlad
  • 43,099
  • 13
  • 111
  • 179