3

I need help to build keras model for classification. I have

Input: 167 points of optical spectrum

Output 11 classes of investigated substance.

But in one data set can be spectre of substance with several substance (for example contains classes 2,3,4). I tried to use categorical_crossentropy, but it is suitable only for non-intersecting classes.

KerasDoc:

Note: when using the categorical_crossentropy loss, your targets should be in categorical format (e.g. if you have 10 classes, the target for each sample should be a 10-dimensional vector that is all-zeros expect for a 1 at the index corresponding to the class of the sample). In order to convert integer targets into categorical targets, you can use the Keras utility to_categorical:

My code:

model = Sequential()
model.add(Dense(64, input_dim=167))
model.add(Dense(32))
model.add(Dense(11))
model.add(Activation('sigmoid'))
model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])

I tried many models but can not get a good result.

utapyngo
  • 6,946
  • 3
  • 44
  • 65
Mogost
  • 496
  • 1
  • 5
  • 9

1 Answers1

1

You should probably go well with sigmoid and binary_crossentropy (See here)

PS: This is not your case, but for a categorial_crossentropy, you should ideally use a softmax activation. The softmax outputs things optimized to maximize one class only.

(If anyone would like to complement this answer with a good or better "optimizer", feel free).

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Daniel Möller
  • 84,878
  • 18
  • 192
  • 214
  • I tried use binary_crossentropy but on every fit action if check on random spectre i get prediction [ 0.23598771 0.24065545 0.20274314 0.20727901 0.21786793 0.20546967 0.24399549 0.23881513 0.22483987 0.24453731 0.2304628 ] But valid is [0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0]. I cannot get any truth-like result. – Mogost May 24 '17 at 16:49
  • You expect to get true values from "random spectre"? If you use `verbose=1` (or 2) in the fit method, you will be able to see the "loss" of your model decreasing. Also, you can use `epochs=someNumber` to repeat the training several times. If the loss decreases, the model is training ok, if it doesn't, there is a problem. I think that 167 data samples is too few for your model to learn things properly. I suggest some kind of data augmentation, such as creating composite (not random) spectres with different combinations of different substances you know. – Daniel Möller May 24 '17 at 17:19
  • One thing you must have in mind for small data samples: you can make your model train and predict them correctly, but if they're too few, there is a chance of your modeling be memorizing them instead of understanding them (it's called overfittin). If a model overfits, it seems great with training data, but different data not contained in training data won't have good results. – Daniel Möller May 24 '17 at 17:23