1

I've written an Adaline Neural Network. Everything that I have compiles, so I know that there isn't a problem with what I've written, but how do I know that I have to algorithm correct? When I try training the network, my computer just says the application is running and it just goes. After about 2 minutes I just stopped it.

Does training normally take this long (I have 10 parameters and 669 observations)? Do I just need to let it run longer?

Hear is my train method

public void trainNetwork()
{
    int good = 0;

    //train until all patterns are good.
    while(good < trainingData.size())
    {
        for(int i=0; i< trainingData.size(); i++)
        {
            this.setInputNodeValues(trainingData.get(i));
            adalineNode.run();

            if(nodeList.get(nodeList.size()-1).getValue(Constants.NODE_VALUE) != adalineNode.getValue(Constants.NODE_VALUE))
            {
                adalineNode.learn();
            }
            else
            {
                good++;
            }
        }
    }
}

And here is my learn method

public void learn()
{
    Double nodeValue = value.get(Constants.NODE_VALUE);
    double nodeError = nodeValue * -2.0;
    error.put(Constants.NODE_ERROR, nodeError);
    BaseLink link;
    int count = inLinks.size();
    double delta;

    for(int i = 0; i < count; i++)
    {
        link = inLinks.get(i);
        Double learningRate = value.get(Constants.LEARNING_RATE);
        Double value = inLinks.get(i).getInValue(Constants.NODE_VALUE);
        delta = learningRate * value * nodeError;
        inLinks.get(i).updateWeight(delta);
    }
}

And here is my run method

public void run()
{
    double total = 0;

    //find out how many input links there are
    int count = inLinks.size();

    for(int i = 0; i< count-1; i++)
    {
        //grab a specific link in sequence
        BaseLink specificInLink = inLinks.get(i);
        Double weightedValue = specificInLink.weightedInValue(Constants.NODE_VALUE);
        total += weightedValue;
    }

    this.setValue(Constants.NODE_VALUE, this.transferFunction(total));
}

These functions are part of a library that I'm writing. I have the entire thing on Github here. Now that everything is written, I just don't know how I should go about actually testing to make sure that I have the training method written correctly.

I asked a similar question a few months ago.

Community
  • 1
  • 1
j.jerrod.taylor
  • 1,120
  • 1
  • 13
  • 33
  • Training networks can potentially take a very long time. I suggest adding some temporary print statements / logging so you can monitor the progress, or running it in a profiler to see where it's spending most of its time. – Jason C Aug 07 '13 at 17:55
  • 2
    _"Everything that I have compiles, so I know that there isn't a problem with what I've written"_ -- not true. I can write lots of code that compiles fine and does nothing useful. Successful compilation is just the first step. – Jim Garrison Aug 07 '13 at 23:15
  • @JimGarrison True. What I meant by that is I'm pretty sure that if it isn't working, I'm close to having a correct algorithm. I think I'd have to make small tweeks instead of a major overhaul. – j.jerrod.taylor Aug 08 '13 at 11:20

2 Answers2

3

Ten parameters with 669 observations is not a large data set. So there is probably an issue with your algorithm. There are two things you can do that will make debugging your algorithm much easier:

  1. Print the sum of squared errors at the end of each iteration. This will help you determine if the algorithm is converging (at all), stuck at a local minimum, or just very slowly converging.

  2. Test your code on a simple data set. Pick something easy like a two-dimensional input that you know is linearly separable. Will your algorithm learn a simple AND function of two inputs? If so, will it lean an XOR function (2 inputs, 2 hidden nodes, 2 outputs)?

bogatron
  • 18,639
  • 6
  • 53
  • 47
0

You should be adding debug/test mode messages to watch if the weights are getting saturated and more converged. It is likely that good < trainingData.size() is not happening.

Based on Double nodeValue = value.get(Constants.NODE_VALUE); I assume NODE_VALUE is of type Double ? If that's the case then this line nodeList.get(nodeList.size()-1).getValue(Constants.NODE_VALUE) != adalineNode.getValue(Constants.NODE_VALUE) may not really converge exactly as it is of type double with lot of other parameters involved in obtaining its value and your convergence relies on it. Typically while training a neural network you stop when the convergence is within an acceptable error limit (not a strict equality like you are trying to check).

Hope this helps

Anshul
  • 680
  • 1
  • 5
  • 19