-6

I built this ML model in Azure ML studio with 4 features including a date column.

Trying to predict if the price is going to be higher tomorrow than it is today. Higher = 1, not higher = 0

It is a Two class neural network (with a Tune model hyperparameters).

When I test it I expect to get a answer between 0 - 1 which I do. The problem comes when I change the feature from 1 to 0. And get almost a similar answer.

I thought that if 1 = a score probabilities of 0.6 Then a 0 (with the same features) should give a score of 0.4

Azure ML output

A snapshot of the data (yes I know I need more) enter image description here

Model is trained/tuned on the "Over5" feature, and I hope to get an answer from the Two class neural network module in the range between 0 -1.

The Score module also produce results between 1 and 0. Everything looks to be correct.

I changed normalization method (after rekommendation from commenter) but it does not change the output much.

Everything seems to be in order but my goal is to get a prediction of the likelihood that a day would finish "Over5" and result in a 1.

When I test the model by using a "1" in the Over5 column I get a prediction of 0.55... then I tested the model with the same settings only changing the 1 to a 0 and I still get the same output 0.55...

I do not understand why this is since the model is trained/tuned on the Over5 feature. Changing input in that column should produce different results?

Alexander Johansson
  • 103
  • 10

1 Answers1

2

Outputs of a neural network are not probabilities (generally), so that could be a reason that you're not getting the "1 - P" result you're looking for.

Now, if it's simple logistic regression, you'd get probabilities as output, but I'm assuming what you said is true and you're using a super-simple neural network.

Also, what you may be changing is the bias "feature", which could also lead to the model giving you the same result after training. Honestly there's too little information in this post to say for certain what's going on. I'd advise you try normalizing your features and trying again.

EDIT: Do you know if your neural network actually has 2 output nodes, or if it's just one output node? If there are two, then the raw output doesn't matter quite as much as which node had the higher output. If it's just one, I'd look into thresholding it somewhere (like >0.5 means the price will rise, but <=0.5 means the price will fall, or however you want to threshold it.) Some systems used in applications where false positives are more acceptable than false negatives threshold at much lower values, like 0.2.

cyniikal
  • 134
  • 1
  • 6
  • Thank you, I changed the normalization but it produced the same results. Could you explain what you mean with the bias feature? – Alexander Johansson Aug 22 '18 at 11:58
  • All we mean by bias "feature" is some constant that we're adding onto each layer into the network allowing us to shift the activation values to the left or right. https://stackoverflow.com/questions/2480650/role-of-bias-in-neural-networks has an excellent post covering what the bias is used for. I can see now, though, that you're not changing the bias term, you're changing the over5 feature. – cyniikal Aug 22 '18 at 12:16