0

I am studying deep learning and as an assignment, I am doing a classification project, which has 17k records with 14 features and a target variable that have 11 classes. enter image description here

I tried to train a simple neural network

# define the keras model
model1 = keras.Sequential()
model1.add(keras.layers.Dense(64, input_dim=14, activation='relu'))
model1.add(keras.layers.Dense(128, activation='relu'))
model1.add(keras.layers.Dense(64, activation='relu'))  
model1.add(keras.layers.Dense(1, activation='softmax'))

# compile the keras model
model1.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

# fit the keras model on the dataset
performance1 = model1.fit(x_train, y_train, epochs=100, validation_split=0.2)

But the problem here is I am getting the same accuracy for each epoch, it doesn't seem that model is even learning. enter image description here

I tried to research this problem and found some similar problems on StackOverflow like this question and tried following things

  • Applied StandardScaler
  • Increased/Decreased the hidden layer and neurons
  • Added dropout layer
  • Changed the optimizers, loss, and activation function
  • I also tried to batch_size

But none of them worked, of course, the accuracy was different in the different trials (but has the same problem).

Few of trials are as follows:

# define the keras model
model1 = keras.Sequential()
model1.add(keras.layers.Dense(64, input_dim=14, activation='sigmoid'))
model1.add(keras.layers.Dense(128, activation='sigmoid'))
model1.add(keras.layers.Dense(64, activation='sigmoid'))  
model1.add(keras.layers.Dense(1, activation='softmax'))

sgd = keras.optimizers.SGD(lr=0.01)
# compile the keras model
model1.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

# define the keras model
model1 = keras.Sequential()
model1.add(keras.layers.Dense(64, input_dim=14, activation='relu'))
model1.add(keras.layers.Dense(128, activation='relu'))
model1.add(keras.layers.Dropout(0.2))
model1.add(keras.layers.Dense(64, activation='relu')) 
model1.add(keras.layers.Dropout(0.2))
model1.add(keras.layers.Dense(1, activation='softmax'))

# compile the keras model
model1.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

I don't know what's the problem here. Please let me know if you require more details to process this. And please don't close this question I know this question stands a chance to marked as a duplicate question but trust me I tried many things which I can understand as a beginner.

Darkstar Dream
  • 1,649
  • 1
  • 12
  • 23
  • What does your x_train & y_train look like? – TC Arlen Aug 12 '21 at 04:28
  • @TCArlen The above dataset was separated as features(contains all columns except Class col) and label(contain Class col) and then it was further splitted by sklearn train_test_split – Darkstar Dream Aug 12 '21 at 04:43

1 Answers1

1

The problem is that the softmax should be applied on an output array to get probabilities and that output array from the model should represent the logits for each target class. hence you would have to change this line

model1.add(keras.layers.Dense(1, activation='softmax'))
# TO
model1.add(keras.layers.Dense(df['Class'].nunique(), activation='softmax'))

EDIT:

# Let's say you have 11 unique values in your class then you last layer will become
model1.add(keras.layers.Dense(11, activation='softmax'))

# Now your loss will be
model1.compile(loss=tf.keras.loss.SparseCategoricalCrossentropy(), optimizer='adam', metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])
Abhishek Prajapat
  • 1,793
  • 2
  • 8
  • 19