1

I've trained a caffemodel with GoogLeNet. During testing I have a very high accuracy:

I0122 06:00:54.384351 2039975936 solver.cpp:409]     Test net output #0: loss1/loss1 = 0.433825 (* 0.3 = 0.130148 loss) 
I0122 06:00:54.385201 2039975936 solver.cpp:409]     Test net output #1: loss1/top-1 = 0.8764 
I0122 06:00:54.385234 2039975936 solver.cpp:409]     Test net output #2: loss1/top-5 = 0.969 
I0122 06:00:54.385243 2039975936 solver.cpp:409]     Test net output #3: loss2/loss1 = 0.327197 (* 0.3 = 0.0981591 loss) 
I0122 06:00:54.385251 2039975936 solver.cpp:409]     Test net output #4: loss2/top-1 = 0.8918 
I0122 06:00:54.385256 2039975936 solver.cpp:409]     Test net output #5: loss2/top-5 = 0.984601 
I0122 06:00:54.385262 2039975936 solver.cpp:409]     Test net output #6: loss3/loss3 = 0.304042 (* 1 = 0.304042 loss) 
I0122 06:00:54.385268 2039975936 solver.cpp:409]     Test net output #7: loss3/top-1 = 0.9228 
I0122 06:00:54.385273 2039975936 solver.cpp:409]     Test net output #8: loss3/top-5 = 0.9768

No I have a python classifier which looks like this:

caffe.Classifier(MODEL_FILE, PRETRAINED,
                       mean=np.load('train_image_mean.npy').mean(1).mean(1),
                       channel_swap=(2, 1, 0),
                       raw_scale=255,
                       image_dims=(256, 256))

I ran the classifier through all my validation data. The accuarcy is very high. But I get some "nan" probability values for some input images. What's the reason for that? What does "nan" mean? Is it "I did not recognize any class"?

Edit: This question is not a duplication since it refers to the classification and not to the training

Thank you.

Community
  • 1
  • 1
Soccertrash
  • 1,830
  • 3
  • 28
  • 48
  • 1
    nan is an acronym for Not A Number. it means some mathematical operation in your net did something illegal, usually dividing by zero or values exceeding floating point max number... – Shai Jan 22 '16 at 11:27
  • Okay, is there any chance to get down to the root cause in the net? – Soccertrash Jan 22 '16 at 12:29
  • do you know what input image causes the nan to appear? – Shai Jan 23 '16 at 16:01
  • I found a workaround. Since I'm using oversampling during classification the result is non deterministic. If I retry classification in case of "nan" values one or two times I get correct classification. – Soccertrash Jan 24 '16 at 12:26
  • Possible duplicate of [Common causes of nans during training](http://stackoverflow.com/questions/33962226/common-causes-of-nans-during-training) – Shai Feb 07 '16 at 14:54
  • Regarding duplication: there can be many reasons for "nan"s. Some of them are unique to training phase, but some are common for both train and test. Therefore, I still think this question should be marked as duplicate. – Shai Feb 08 '16 at 15:58
  • I came across this phenomenon, too and in my case it turned out to be an error on my side where the transformer that preprocesses the image data before classification assumed it would get image data in the 0...1 range but I was passing in image data in the 0...255 range. – Quasimondo Sep 16 '16 at 09:16

0 Answers0