1

I'm new to deep learning, and machine learning in general. I was trying to work with the California Housing Prices dataset by passing each of the 8 features to a 5 layer network and training on the output of a price. I use MSE loss and accuracy as metric with 2000 epochs. The loss starts off as understandably large, reduces to a certain a degree and gets stuck around the same value. The accuracy is stuck at 0 for all of 2000 epochs.

I have seen a few solutions online that do things like divide the total rooms with the number of households to find the average number of rooms per household. I see that doing such feature engineering would help the model converge faster, but I had hoped the model to converge with no feature engineering at all.

From what I understood, Neural Networks are function approximators such that it builds a function from input to output of the dataset. Ideally I hoped that it would also find complex features like the ones manually computed in the online solutions. Am I wrong is having these expectations? What is the reason that the model isn't converging?

train_data = pd.read_csv('./sample_data/california_housing_train.csv', sep=',')
test_data = pd.read_csv('./sample_data/california_housing_test.csv', sep=',')

model = tf.keras.models.Sequential([
  layers.Dense(8),
  layers.Dense(5, activation=tf.nn.relu),  
  layers.Dense(7, activation=tf.nn.relu),  
  layers.Dense(5, activation=tf.nn.relu),
  layers.Dropout(0.2),
  layers.Dense(1)
])

model.compile(optimizer='adam', loss='mean_squared_error', shuffle=True, metrics=['accuracy'])

model.fit(train_data.values[:, 0:8], train_data.values[:, 8], epochs=2000)

Output Output

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Uzair
  • 603
  • 1
  • 5
  • 11
  • 1
    This sounds like a regression problem. You can't measure accuracy - only error. Can you show us the output of your training? – rayryeng Jun 23 '19 at 05:41

2 Answers2

4

Accuracy is not a valid metric for Regression problem. What function defines accuracy in Keras when the loss is mean squared error (MSE)?

Also, please normalize the input data using Min-max or zero-mean/unit variance normalization. Moreover the range of the output data(order of 10,000) to learn is very large, so, you can divide the output value by 10,000 (during prediction you can multiply this value back). These changes will help the network converge faster. The capacity of the network may need to be increased as well.

Different types of normalization for numeric data: https://developers.google.com/machine-learning/data-prep/transform/normalization

Manoj Mohan
  • 5,654
  • 1
  • 17
  • 21
4

You are solving a regression problem here, so accuracy cannot be used as a metric for evaluating your model. Instead, you could use performance measure like mae , mape, mse which are more suitable for evaluating a model which predicts a continuous target variable.

Moreover I wouldn't say that your model is not converging, if you look at the loss it is reducing constantly, so there is no problem with convergence, you just need to change your evaluation measure to something which I have mentioned above.

Please refer to How to Use Metrics for Deep Learning with Keras in Python or Usage of metrics for more details on how to implement these performance measures.

Sunderam Dubey
  • 1
  • 11
  • 20
  • 40
Parthasarathy Subburaj
  • 4,106
  • 2
  • 10
  • 24
  • Thanks for answering. Yes, the loss decreases down to a certain value, but then just stagnates going up and down around the same value. I see why my accuracy is 0, but why doesn't the loss keep on decreasing then? – Uzair Jun 23 '19 at 16:04
  • It's all in your data. You need to normalize it as the dynamic range between features is wildly different. Take a look at the other answer. – rayryeng Jun 23 '19 at 17:10
  • You are trying to predict the housing prices which I think are measured in a very high range (something in 100,000s i guess), and you have decided to measure your loss using `mean_squared_error` which is very much sensitive to the range in which you measure your variables. Please refer to (https://stackoverflow.com/questions/56689243/is-it-acceptable-to-scale-target-values-for-regressors/56690991#56690991). Or just try normalizing your inputs to have a better picture of what is happening exactly. – Parthasarathy Subburaj Jun 24 '19 at 05:51