0

I've been working on a simple neural network.

It takes in a data set with 3 columns, if the first column's value is a 1, then the output should be a 1. I've provided comments so it is easier to follow.

Code is as follows:

import numpy as np
import random

def sigmoid_derivative(x):
    return x * (1 - x)

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

def think(weights, inputs):
    sum = (weights[0] * inputs[0]) + (weights[1] * inputs[1]) + (weights[2] * inputs[2])
    return sigmoid(sum)

if __name__ == "__main__":

    # Assign random weights
    weights = [-0.165, 0.440, -0.867]

    # Training data for the network.
    training_data = [
        [0, 0, 1],
        [1, 1, 1],
        [1, 0, 1],
        [0, 1, 1]
    ]

    # The answers correspond to the training_data by place,
    # so first element of training_answers is the answer to the first element of training_data
    # NOTE: The pattern is if there's a 1 in the first place, the result should be a one
    training_answers = [0, 1, 1, 0]

    # Train the neural network
    for iteration in range(50000):
        # Pick a random piece of training_data
        selected = random.randint(0, 3)

        training_output = think(weights, training_data[selected])

        # Calculate the error
        error = training_output - training_answers[selected]

        # Calculate the adjustments that need to be applied to the weights
        adjustments = np.dot(training_data[selected], error * sigmoid_derivative(training_output))

        # Apply adjustments, maybe something wrong is going here?
        weights += adjustments

    print("The Neural Network has been trained!")

    # Result of print below should be close to 1
    print(think(weights, [1, 0, 0]))

The result of the last print should be close to 1, however it is not?

I have a feeling that I'm not adjusting the weights correctly.

Crisp Apples
  • 267
  • 1
  • 2
  • 13
  • You're not adding any bias into your network. So right now you have `sigmoid(y=mx)`, but really it should be `sigmoid(y=mx+b)`. Even without adding a weight to you bias, I can change it to `(weights[0] * inputs[0] + 1) + (weights[1] * inputs[1] + 1) + (weights[2] * inputs[2] + 1)` and get an answer of 0.938 – Chrispresso Apr 15 '20 at 22:36
  • @Chrispresso did what you suggested, but now when I change one of the other columns in the print statement it gives me .99, for example `print(think(weights, [0, 1, 0]))` – Crisp Apples Apr 15 '20 at 22:45
  • This looks like the exact same data, and a very similar implementation, to this question: https://stackoverflow.com/questions/61232327/neural-network-cost-doesnt-decrease/61239137 – mcskinner Apr 15 '20 at 22:49
  • You need to introduce weights for the bias and perform backprop on those as well. – Chrispresso Apr 15 '20 at 22:51
  • @Chrispresso weights for the bias? Now I'm a bit lost lol. Any chance you could provide the full code? – Crisp Apples Apr 15 '20 at 22:51

0 Answers0