0

I ran caffe and got this output:
enter image description here

who can tell me what is the problem?
I will really appreciate!!

Shai
  • 111,146
  • 38
  • 238
  • 371
魏嘉毅
  • 109
  • 1
  • 8
  • I did nothing special . I just made a CNN model called NN2 with CASIA-WebFace database. And this was my first time run my own net. – 魏嘉毅 Mar 23 '16 at 09:14

1 Answers1

0

It seems like one (or more) of your label values are invalid, see this PR for information:

If you have an invalid ground truth label, "SoftmaxWithLoss" will silently access invalid memory [...] The old check only worked in DEBUG mode and also only worked for CPU.

Make sure your prediction vector length matches the number of labels you try to predict.

From your comments, it seems like you have labels in the range 0..10575, but on the other hand, your classification layer, "fc7" only predicts probabilities for 1000 classes. Thus, "SoftmaxWithLoss" layer tries to compute the loss for predicting label l>1000, and access memory outside the probability array, resulting with a segmentation fault.

Community
  • 1
  • 1
Shai
  • 111,146
  • 38
  • 238
  • 371
  • it is sure that there is nothing wrong with label. Label is made by procedure. Do you have more advise for me? thank you very much – 魏嘉毅 Mar 23 '16 at 09:53
  • @魏嘉毅 what is the the `num_output` of the layer feeding the `"SoftmaxWithLoss"` layer? what is the range of ground-truth labels you are feeding the net? – Shai Mar 23 '16 at 09:56
  • the range of the labels is 10575 and the loss layer dose not have the param(num_output). layer { name: "SoftMax" type: "SoftmaxWithLoss" bottom: "fc7" bottom: "label" top: "SoftMax" } Did you mean that the two numbers should be the same one? – 魏嘉毅 Mar 23 '16 at 10:05
  • @魏嘉毅 what is the `num_output` of layer `"fc7"`? – Shai Mar 23 '16 at 10:38
  • well,the num_output of fc7 is 1000 – 魏嘉毅 Mar 23 '16 at 10:43
  • 1
    should the num_output of "fc7" cover the range of labels? – 魏嘉毅 Mar 23 '16 at 10:45
  • @魏嘉毅 that's it then: `"fc7"` outputs a probability vector over 1000 possible classes (you tell me what classes these are), but in `"SoftmaxWithLoss"` you are trying to access probabilities of classes beyod these 1000. You need `"fc7"` to output a probability vector of len 10575 and re-train `fc7` accordingly. – Shai Mar 23 '16 at 10:45
  • appreciate you so much! I will re-train it. However, there is a "fc6" which is the input of "fc7". So I have another question ,should I set the "num_output" of "fc6" as 10575? And if both of "fc6" and "fc7" have the same num_output, why it is necessary to set two fully-connected layers? – 魏嘉毅 Mar 23 '16 at 10:56
  • @魏嘉毅 setting `num_output` of `fc6` to `10575` will create a weight matrix of size 10575x10575=111,830,625 (!) good luck training this... No, you do not need to change the `num_output` of `fc6`. I'm afraid explaining this is considered "too broad" for stackoverflow threads. – Shai Mar 23 '16 at 11:04
  • so what i need to do is just set the num_output to 10575? do you mean that? – 魏嘉毅 Mar 23 '16 at 11:12
  • And the num_output of "fc6" can maintain as "1000"? – 魏嘉毅 Mar 23 '16 at 11:14
  • @魏嘉毅 yes, you only need to change `num_output` of `fc7` and leave `fc6` as is. – Shai Mar 23 '16 at 11:26
  • 1
    thank you! This is my first time asking at stackoverflow and your support helps me so so much! What'a more , I am a Chinese . Hence, i have not thought i could get a suitable answer so smoothly – 魏嘉毅 Mar 23 '16 at 11:35