3

Currently I'm using VGG16 + Keras + Theano thought the Transfer Learning methodology to recognize plants classes. It works just fine and gives me a good accuracy. But the next problem I'm trying to solve - is to find a way of identifying if an input image contains plant at all. I don't want to have another one classifier that will do it, because it's not really efficiently.

So I did some search and have found that we can get activations from the latest model layer (before activation layer) and analyze it.

from keras import backend as K

model = util.load_model() # VGG16 model
model.load_weights(path_to_weights)

def get_activations(m, layer, X_batch):
    x = [m.layers[0].input, K.learning_phase()]
    y = [m.get_layer(layer).output]
    get_activations = K.function(x, y)
    activations = get_activations([X_batch, 0])

    # trying to get some features from activations
    # to understand how can we identify if an image is relevant
    for l in activations[0]:
        not_nulls = [x for x in l if x > 0]

        # shows percentage of activated neurons
        c1 = float(len(not_nulls)) / len(l)
        n_activated = len(not_nulls)
        print 'c1:{}, n_activated:{}'.format(c1, n_activated)

    return activations

get_activations(model, 'the_latest_layer_name', inputs)

From the above code I've noticed that when we have very irrelevant image, the number of activated neurons is bigger than for images that contain plants:

  1. For images that was using for model training, number of activated neurons 19%-23%
  2. For images that contain unknown plants species 20%-26%
  3. For irrelevant images 24%-28%

It's not really a good feature to understand if an image relevant as percentage values are intersect.

So, is there a good way to resolve this issue?

Arsey
  • 281
  • 3
  • 10
  • I think, a better way is to check which nuerons got activated for relevant images rather than the percentage of neurons. Is there a lot of overlap? – Krishna Kishore Andhavarapu Oct 28 '16 at 11:16
  • as Krishan said the better way is to consider them as features and check if there is any correlation within the two classes " clustering". another classifier might be added also to a specific layer to decide on yes or no. – Feras Oct 29 '16 at 22:21

1 Answers1

0

Thanks to Feras's idea in the comment above. After some trials, I've come up with the ultimate solution that allows solving this problem with accuracy up to 99.99%.

Steps are:

  1. Train your model on a dataset;
  2. Store activations (see method above how to get them) by predicting relevant and non-relevant images using trained model from the previous step. You should get activations from the penultimate layer. For VGG16 it's the last of two Dense(4096), for InceptionV3 - an extra penultimate Dense(1024) layer, for resnet50 - an extra penultimate Dense(2048) layer.
  3. Solve a binary problem using stored activations data. I've tried a simple flat NN and Logistic Regression. Both were good in accuracy (flat NN was a bit more accurate), but I've chosen the Logistic Regression as it's simpler, faster and consumes less memory and CPU/GPU.

This process should be repeated each time after your model retrained as each time the final weights for CNN are different and what was working previously, will be different next time.

So as result we have another small model for solving the problem.

Arsey
  • 281
  • 3
  • 10