caffe: A CNN with two loss layers and two different label sets

Question

I am interested to have a deep CNN with two loss layers, each of them having its own separate label set. For example, imagine a 10 layer CNN, loss1 is located in 5th layer using label1, then loss2 in the last layer using label2.

Note: label1 (loss1) can be considered as pre-processing step, in order to ease the task of the CNN to solve label2 (loss2 i.e. real label).

Question: as far as i know, caffe loss layer automatically uses "label" (in my .lmdb dataset). how to make it to use "label1" in loss1 and "label2" in the loss2?

Update: so i took Shai's advice to use HDF5. here is my progress so far (giving only the input and output layers):

layer {
  name: "data"
  type: "HDF5Data"
  top: "data"
  top: "label1"
  top: "label2"
  include {
    phase: TEST
  }
  hdf5_data_param {
    source: "data/2048_1e3_0.00/2048_1e3_0.00_s_val_list.txt"
  }
}
layer {
  name: "data"
  type: "HDF5Data"
  top: "data"
  top: "label1"
  top: "label2"
  include {
    phase: TRAIN
  }
  hdf5_data_param {
    source: "data/2048_1e3_0.00/2048_1e3_0.00_s_train_list.txt"
  }

...

layer {
  name: "loss1"
  type: "EuclideanLoss"
  bottom: "ampl"
  bottom: "label1"
  top: "loss1"
}

...

layer {
  name: "loss2"
  type: "EuclideanLoss"
  bottom: "ampl"
  bottom: "label2"
  top: "loss2"
}

Am I in the right direction?

thanks Shai. the "this answer" link you provided is not addressing the question about using multiple loss using multiple labels. so i decided to make my question more clear by "update". also, I provided a part of code, to ask if that is how I should do it or not. what is wrong with that? — Nima, Mar 28 '18 at 15:59
The answer I [linked to](https://stackoverflow.com/a/31808324/1714410) shows how to create HDF5 data for caffe. You should modify it have multiple labels instead of just `X` and `Y`. And yes, the direction you took seems to lead to a good solution. — Shai, Mar 28 '18 at 16:03

Shai · Accepted Answer · 2018-03-28T05:41:03.320

AFAIK Caffe stores binary Datums in lmdb. The way Datum is defined restricts it to a single integer label per input image. Therefore it can be very tricky to use "Data" layer with lmdb in your scenario.

On the other hand you can use "HDF5Data" layer instead: this layer is much more flexible and can easily provide with more than a single label per input image. There are several examples here at SO on how to use this input layer, see, e.g. this answer.

caffe: A CNN with two loss layers and two different label sets

1 Answers1