Caffe: Loading images and labels within python for fine-tuning

Question

I am having some problems when I try to load a batch of images + labels within python and try use it to train a network. I am working with pair of images, that I transform into one (for example, by averaging equivalent pixels from both images) and then I feed it to the network. As the number of pairs is too big (combination of the individual images two by two) to store all in memory, I am creating each batch in each iteration and I would like to give it to the network.

I am instantiating the network this way:

solver = caffe.get_solver(path_to_solver_file)

where the input network layers are defined with:

layer {
  name: "data"
  type: "ImageData"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  image_data_param { 
    batch_size: 1
    new_height: 227
    new_width: 227
  }
}
layer {
  name: "data"
  type: "ImageData"
  top: "data"
  top: "label"
  include {
    phase: TEST
  }
  image_data_param {
    batch_size: 1
    new_height: 227
    new_width: 227
  }
}

In my earlier experiments, I had a source parameter inside image_data_param, where I would pass a file with my images and labels for both training and test. However, as I want to load them within python, I removed the source parameter, but got the following error:

0830 17:01:49.014819 1967923200 layer_factory.hpp:77] Creating layer data

I0830 17:01:49.014858 1967923200 net.cpp:91] Creating Layer data

I0830 17:01:49.014868 1967923200 net.cpp:399] data -> data

I0830 17:01:49.014890 1967923200 net.cpp:399] data -> label

I0830 17:01:49.014910 1967923200 image_data_layer.cpp:38] Opening file 

I0830 17:01:49.014935 1967923200 image_data_layer.cpp:53] A total of 0 images.

Segmentation fault: 11

I haven't got to this point yet, but after I'm able to instantiate the network, I was going to load the batch and perform one step of the SGD optimization using:

net.blobs["data"].data[...] = images
net.blobs["label"].data[...] = labels
net.step(1)

I have searched for examples and tutorials (for example, here and here) that perform fine-tuning and testing using python, however the great majority only discuss the forward passes during a test phase, whereas the ones that fine-tune a network define the training data (labels and images) using the source parameter, and not by loading directly from the python interface.

Check out this answer: http://stackoverflow.com/a/39097123/5465000 — Amir, Sep 01 '16 at 01:56

score 1 · Accepted Answer · edited May 23 '17 at 12:34

1

The layer ImageData can only read images from a text file which holds the address of images on each line. You cannot use an ImageData layer and feed in images on the fly.

If you are interested in loading images on the fly, checkout this link. I suggest using the deploy version (3rd option explained in that post), which is way handier and more flexible.

When using the deploy version, you can set the input data for your network like this:

x = data;
y = labels;
solver.net.blobs['data'].data[...] = x
net.blobs['label'].data[...] = y

Then you can either call solver.net.forward() to only see the outputs; or you can call solver.net.step(1) to run a forward and a backward (train for 1 iteration).

edited May 23 '17 at 12:34

Community

1
1

answered Sep 01 '16 at 17:54

Amir

2,259
1
19
29

Hey, @AHA, thanks for your answer. But I have two questions: (1) Using the deploy version, may I still use it to fine-tune my network? (2) If so, how would I pass the data to these layers? Is it similar to what I've done in the post? – rafaspadilha Sep 02 '16 at 01:24
Yes, similar to what you had done in your post. I edited and extended my answer to make that clear. – Amir Sep 02 '16 at 17:37
That good news! It seems I won't have to change a lot in my code to do it. I'll give it a try and I'll report back! Thanks again @AHA! – rafaspadilha Sep 02 '16 at 20:49

Caffe: Loading images and labels within python for fine-tuning

1 Answers1