calculate equation with non linear data

Question

I'm making my first steps with Tensorflow.js and I need to calculate the equation

y = [(x * 100) / a]/100

I.E. given 4 tensors like:

[1,1,1,0], [2,2,2,1], [0,0,1,0], [0,2,2,1]

the sums of all values for each tensor would be:

3, 7, 1, 5

the sums of those values would be:

and the equation above would be:

y = [(3 * 100) / 15]/100
y = [(7 * 100) / 15]/100
y = [(1 * 100) / 15]/100
y = [(5 * 100) / 15]/100

so the output tensor should be:

[0.19], [0.44], [0.06], [0.31]

I made the code below where I tried to train a model to solve the equation, but the results are far from being acceptable. I even tried generating 60 couples of input and output examples, augmenting epochs of training to 50k and augmenting the number of units of the input layer, but the results seem to be even worse. Can you give me some help? Where am I mistaking? Thank you!

<script>
    async function predictOutput() {

        const model = tf.sequential();
        //config for the hidden layer
        const config_hidden = {
          inputShape:[4],
          activation:'sigmoid',
          units:4
        }
        //config for the output layer
        const config_output={
          units:1,
          activation:'sigmoid'
        }
        //defining the hidden and output layer
        const hidden = tf.layers.dense(config_hidden);
        const output = tf.layers.dense(config_output);
        //adding layers to model
        model.add(hidden);
        model.add(output);
        //define an optimizer
        const optimize=tf.train.sgd(0.1);
        //config for model
        const config={
        optimizer:optimize,
        loss:'meanSquaredError'
        }
        //compiling the model
        model.compile(config);

        //Dummy training data
        const x_train = tf.tensor([
        [1,0,0,3], [0,0,3,0], [1,0,0,0], [0,1,0,4],
        [0,0,0,1], [2,0,2,1], [2,4,1,0], [0,2,0,1],
        [1,1,1,0], [2,2,2,1], [0,0,1,0], [0,2,2,1],
        [1,0,0,0], [0,1,0,0], [1,1,1,0], [2,2,2,2],
        [2,5,7,9], [2,1,0,10], [22,5,7,9], [2,0,3,1],
        [1,1,1,1], [2,2,2,2], [0,5,8,1], [5,5,8,1],
        [3,4,1,5], [1,0,3,1], [5,5,1,0], [4,2,6,0],
        [1,0,0,0], [1,1,2,1], [1,3,2,1], [1,2,0,0],
        [1,0,0,2], [0,0,0,7], [0,1,0,0], [5,0,0,0],
        [0,4,0,0], [1,0,7,0], [3,2,8,1], [0,10,9,0]
        ]);

        //Dummy training labels
        const y_train = tf.tensor([
        [0.31], [0.23], [0.08], [0.38],
        [0.07], [0.31], [0.44], [0.18],
        [0.19], [0.44], [0.06], [0.31],
        [0.08], [0.08], [0.23], [0.61],
        [0.27], [0.15], [0.51], [0.07],
        [0.09], [0.18], [0.31], [0.42],
        [0.32], [0.12], [0.27], [0.29],
        [0.07], [0.31], [0.44], [0.18],
        [0.19], [0.44], [0.06], [0.31],
        [0.09], [0.18], [0.31], [0.42]
        ]);

        //Dummy testing data
        const x_test = tf.tensor([
            [1,0,0,1], [0,1,1,0], [2,0,1,2], [0,0,0,1]
        ]);

        // expected result: [0.20], [0.20], [0.50], [0.10]


        await model.fit(x_train, y_train, {
            batchSize: 1,
            epochs: 5000
        });

        // Test the model and display output 
        document.getElementById("output").innerText = model.predict(x_test);
    }
    predictOutput();
</script>

Some things to note: 1) you have very few samples of training data 2) if you know the explicit function, what are you trying to do with modelling it via a NN? Not trying to be rude with the second question, just want to know what problem you're trying to address — IanQ, Nov 09 '18 at 19:18
Hi, thanks for your comment. 1) As I wrote in the question, I generated 60 couples of inputs and outputs but the results don't seem to be better. 2) I already have written a working code whithout using Tensorflow, I just was asked to do the same with the Tensorflow. — ing.gambardella, Nov 12 '18 at 08:11

edkeveked · Answer 1 · 2018-11-12T15:00:11.117

If the linear equation is known, i.e y = 1/15x, then the output can be calculated directly using the following function

const calc = (t) => t.add(tf.scalar(1/15))

If the slope a and the bias b are now known in advance and need to be computed, a perceptron neural network can be used. The inputShape of the model is wrong since the prediction is to be done for a single value. It should be [1] instead of 4. As for the x_test, it should look like an array of array of length 1 which means the following:[[1], [2], [3], [56], [49]]

Also using a smaller learning rate might lead to better accuracy

EDIT:

If the model has to predict on the sum of the inputs, then, inputShape should be [4]. Your input values are small and that is affecting the error loss using a perceptron model. Before giving the data to the model, one can process the input values multiplying by 10 for instance. As a result, the model prediction will be 10 times the correct values.

const model = tf.sequential()
model.add(tf.layers.dense({inputShape: [4], units: 1 }))
model.add(tf.layers.dense({units: 1}))
const x_train = tf.tensor([
        [1,0,0,3], [0,0,3,0], [1,0,0,0], [0,1,0,4], [0,0,0,1]]);
const y = tf.tensor([
        [0.31], [0.23], [0.08], [0.38],[0.07]
        ]);

const y_train = y.mul(tf.scalar(10))
const optimizer = tf.train.sgd( 0.01 )
model.compile({optimizer: optimizer, loss: 'meanSquaredError' })
model.fit(x_train, y_train, {epochs: 1000, 
      callbacks: {
      onEpochEnd: (epoch, log) => {
        console.log(epoch, log.loss);
        if (epoch === 100 ) {
           model.optimizer.setLearningRate(.001);
        }
      }
    }}).then(() => {
  const y_predict = model.predict(x_train)
  const y_correct = y_predict.div(tf.scalar(10)).print()
})

<html>
  <head>
    <!-- Load TensorFlow.js -->
    <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@0.13.3"> </script>
  </head>

  <body>
  </body>
</html>

Hi, thanks for your answer. I already wrote a working code that calculates the outputs without using the model training and prediction, I just need to do it that way for research working purpose. By your answer I do understand that no matter how many inputs I use to train the model and how many epochs I add to the model.fit function, the prediction will never work. Is that right? I need the input shape to be that form, because I can't sum the inputs before accessing the prediction function. I need the function to understand that it has to sum the inputs in order to make the prediction. — ing.gambardella, Nov 12 '18 at 08:18
@ing.gambardella, if the prediction is to be done on the sum, then the inputShape will be 4. Please consider the edit to the question. And don't forget to upvote the answer and mark it as accepted :) — edkeveked, Nov 12 '18 at 15:03
I tried to modify your code to adapt it to my work, but it doesn't predict the output correctly. The input can have 5 tensor, the problem is that the sum of the output tensors always has to be 100. In your example: [0.31], [0.23], [0.08], [0.38],[0.07] = 107 So I deleted the last tensor both from the input and from the output like this: [1,0,0,3], [0,0,3,0], [1,0,0,0], [0,1,0,4] -> [0.31], [0.23], [0.08], [0.38] It still predicts the right output, but if I try to predict another tensor, it doesn't work: [1,0,0,1], [0,1,1,0], [2,0,1,2], [0,0,0,1] -> should be: [0.20] [0.20] [0.50] [0.10] — ing.gambardella, Nov 13 '18 at 10:34

calculate equation with non linear data

1 Answers1