2

I try to use pretrained model (VGG 16) to DIGITS but I got this error.

ERROR: Check failed: error == cudaSuccess (2 vs. 0) out of memory

and 

conv2_2 does not need backward computation.
relu2_1 does not need backward computation.
conv2_1 does not need backward computation.
pool1 does not need backward computation.
relu1_2 does not need backward computation.
conv1_2 does not need backward computation.
relu1_1 does not need backward computation.
conv1_1 does not need backward computation.
data does not need backward computation.
This network produces output label
This network produces output softmax
Network initialization done.
Solver scaffolding done.
Finetuning from /home/digits/digits/jobs/20161020-095911-9d01/model.caffemodel
Attempting to upgrade input file specified using deprecated V1LayerParameter: /home/digits/digits/jobs/20161020-095911-9d01/model.caffemodel
Successfully upgraded file specified using deprecated V1LayerParameter
Attempting to upgrade input file specified using deprecated input fields: /home/digits/digits/jobs/20161020-095911-9d01/model.caffemodel
Successfully upgraded file specified using deprecated input fields.
Note that future Caffe releases will only support input layers and not input fields.
Check failed: error == cudaSuccess (2 vs. 0)  out of memory

I uploaded deploy.prototxt and VGG_ILSVRC_16_layers.caffemodel and synset_words.txt successfully into DIGITS and tested with my data-set which has two classes.

MD. Nazmul Kibria
  • 1,080
  • 10
  • 21
Farshid PirahanSiah
  • 761
  • 6
  • 22

1 Answers1

2

Sometimes digits-server can not clear the memory. Try with this command if you are using ubuntu:

sudo restart nvidia-digits-server

If this do not work and you face again again the same, you need to reduce the batch_size

MD. Nazmul Kibria
  • 1,080
  • 10
  • 21
  • Thanks. The memory issue is solved but the error remained. I have to change the last layer but when I change the name the process of training does not going well. – Farshid PirahanSiah Oct 20 '16 at 06:43
  • both pretrained and your data set are same? If at least pretrained model is not prepared from similar database your will not get better results – MD. Nazmul Kibria Oct 20 '16 at 06:57
  • I have different dataset with only two classes and I want to use the model for my dataset. I read " fine tuning " and "Transfer Learning" in order to modify and change for my new dataset but I got error in training. – Farshid PirahanSiah Oct 20 '16 at 07:09