3

According to this answer, one should never use more than two hidden layers of Neurons.

According to this answer, a middle layer should contain at most twice the amount of input or output neurons (so if you have 5 input neurons and 10 output neurons, one should use (at most) 20 middle neurons per layer).

Does that mean that all data will be modeled within that amount of Neurons?

So if, for example, one wants to do anything from modeling weather (a million input nodes from data from different weather stations) to simple OCR (of scanned text with a resolution of 1000x1000DPI) one would need the same amount of nodes?

PS.

My last question was closed. Is there another SE site where these kinds of questions are on topic?

Community
  • 1
  • 1

2 Answers2

0

You will likely have overfitting of your data (aka, High Variance). Think of it like this: The more neurons and layers you have gives you more parameters to fit your data better.

Remember that for the first layer node the equation becomes Z = sigmoid(sum(W*x)) The second layer node becomes Z2 = Sigmoid(sum(W*Z))

Look into machine learning class taught at Stanford...its a great online course and good tool as a reference.

Adam
  • 593
  • 7
  • 13
0

More than two hidden layers can be useful in certain architectures such as cascade correlation (Fahlman and Lebiere 1990) and in special applications, such as the two-spirals problem (Lang and Witbrock 1988) and ZIP code recognition (Le Cun et al. 1989).

  • Fahlman, S.E. and Lebiere, C. (1990), "The Cascade Correlation Learning Architecture," NIPS2, 524-532.
  • Le Cun, Y., Boser, B., Denker, J.s., Henderson, D., Howard, R.E., Hubbard, W., and Jackel, L.D. (1989), "Backpropagation applied to handwritten ZIP code recognition", Neural Computation, 1, 541-551.

Check out the sections "How many hidden layers should I use?" and "How many hidden units should I use?" on comp.ai.neural-nets's FAQ for more information.

Franck Dernoncourt
  • 77,520
  • 72
  • 342
  • 501