I am currently having a problem that I haven't been able to figure out for a couple of days with a self-implemented neural network in Java. My network has two input neurons corresponding to x and y coordinates of a pixel in a given greyscale image and one output representing the tone of the pixel. The learning algorithm I'm using is RPROP. The problem I am having is that after numerous iterations of feeding the network with the whole training data set, it converges to a point very far away from the one expected and after analysing the weight structure of the converged network, I could see that all neurons in the hidden layer had the exact same set of input weights. This happens independently of the number of neurons in the hidden layer. Is it caused by the fact that I'm initializing all weights in the network with the same value? I tried randomizing the weights but it didn't make things any better. I am using sigmoidal (tanh) activation functions in all the layers except the output layer. I don't know if I made a bug in implementation or if I misunderstood some part of mathematical description of neural network learning process. Does anyone know what might cause such strange behaviour?
Asked
Active
Viewed 1,354 times
0
-
How would the neural network deduce color from coordinates alone? – endolith Aug 20 '18 at 23:14
2 Answers
0
Try to debug with printouts of what the network is doing. If you have a small number of units, you can print out the activations, deltas, weights and changes to the weights. If you have a lot - you can try and print out the average value of their absolute value.
This may lead you to the problem.
Also - have you tried playing with the learning rate/momentum?
Uri
- 25,622
- 10
- 45
- 72
0
Are you using bias units? I would look up the use of bias units in Neural Networks. Also, if you're doing a simple out of the box implementation, you may want to test iterative results of your work against a known NN library.
nmjohn
- 1,432
- 7
- 16