0

I am new to ANN and ML. I write a simple piece of code using keras to train my ANN. This ANN has 165 inputs and 1 output with multiple observations (regression model). My problem here is that both acc and val_acc are zero, which makes no sense to me. I am not sure what is the root causes, my model setting or my data. And two other problems are 1) if I train this model twice, different results will be obtained; and 2) for the model evaluation, which data sets I should use, train data or test data? Thanks!

model = keras_model_sequential()
model %>% layer_dense(units = 10, activation = 'relu', input_shape = 165)
model %>% layer_dense(units = 6, activation = 'relu')
model %>% layer_dense(units = 1)

summary(model)

model %>% compile(
  loss = "mean_absolute_percentage_error",
  optimizer = optimizer_adam(lr = 0.001, beta_1 = 0.9, beta_2 = 0.999 ),
  metrics = c('accuracy')
)

Fitted_model = model %>% fit(
  x_train, y_train,
  epochs = 50, batch_size = 20,
  validation_split = 0.2
)

score <- model %>% evaluate(
  x_train, y_train
)

cat('Test loss:', score$loss, '\n')
cat('test accuracy:', score$acc, '\n')

y_predict <- model %>% predict(x_test, batch_size = 128)
Harry
  • 331
  • 1
  • 4
  • 14
  • If I set the node # in the final layer as 2 or more, non-zero acc and val_acc will be obtained. But it makes no sense to me at all. – Harry Jul 22 '19 at 15:38

1 Answers1

0

I'm asking my questions here, because i haven't earned the privilege to comment.... To get an answer if it is your data or your model what causes your problem, we need to see the data, or at least get the shape of the data. Also some outputs of the model like accuracy etc. wouldn't hurt.

For your question 1: What do you mean by different results? Normaly you initialize the weights of every layer. If you dont set your kernel_initializer attribute, keras uses 

kernel_initializer ='glorot_uniform'

as seen in the API. https://keras.io/layers/core/ So if you don't set a seed, your weights will be different for every run. That's why you will get different results even if you train with the same data.

Question 2: Normaly you use a subset of your trainings data as validation data. For More information on this you could read this https://towardsdatascience.com/train-validation-and-test-sets-72cb40cba9e7

Overall even if you use data that your model has already seen (your trainings data) your validation accuracy of 0% indicates, that your model didn't really learn while training. This could be because you have a very very small network but many features. You might try to select your featrues more carefully and do a feature selection. Some input can you find here https://towardsdatascience.com/feature-selection-techniques-in-machine-learning-with-python-f24e7da3f36e Also you can increase your network and tune the hyperparameters to get a better result (but i would start with the feature selection). To get an idea of Hyperparameter Tuning this might be helpful: https://www.analyticsvidhya.com/blog/2018/11/neural-networks-hyperparameter-tuning-regularization-deeplearning/

I hope this helps. Feel free to ask if anything was unclear.

Fabian
  • 756
  • 5
  • 12
  • Hi Fabian, thanks for your response.I found a post saying that metric "accuracy" was suitable for classification, not for regression. So I changed the metric. For Q1, your answer makes me clear. For Q2, these links are very informative, I will go through for more details. For me, the concept of train set, validate set and test set are clear. I am not sure which I should choose for "model %>% evaluate()". By the way, are there any good resources for ANN and ML, since I am new and it's hard for me to understand some advanced concepts, like regularization and attributes of optimizer. – Harry Jul 23 '19 at 15:49
  • Here is the link for the post: https://stackoverflow.com/questions/41819457/zero-accuracy-training-a-neural-network-in-keras – Harry Jul 23 '19 at 17:00
  • Hi Harry, i think i have to work on my reading skills a little bit more. But nice that you figured it out by yourself. A book which helped me alot was [this one](https://www.amazon.de/Hands-Machine-Learning-Scikit-Learn-TensorFlow/dp/1491962291) Another good resource could be [this](https://machinelearningmastery.com/how-to-reduce-generalization-error-in-deep-neural-networks-with-activity-regularization-in-keras/). I think you can finde something about optimizers there as well...or you look [here](https://www.dlology.com/blog/quick-notes-on-how-to-choose-optimizer-in-keras/) Hope it helps! – Fabian Jul 26 '19 at 11:52