caffe hdf5 H5LTfind_dataset(file_id, dataset_name_) Failed to find HDF5 dataset

Question

I was using HDF5 as one of the input to feed caffe, the hdf5 file only contains some weight information to put in the sigmoidcrossentropyloss layer so it doesn't contain any label.This error occured:

    I1015 07:08:54.605777 17909 net.cpp:100] Creating Layer weight28
    I1015 07:08:54.605797 17909 net.cpp:408] weight28 -> weight28
    I1015 07:08:54.605834 17909 hdf5_data_layer.cpp:79] Loading list of HDF5 filenames from: /home/zhangyu/codes/unsupervised/data/weight28.txt
    I1015 07:08:54.605926 17909 hdf5_data_layer.cpp:93] Number of HDF5 files: 1
    F1015 07:08:54.608682 17909 hdf5.cpp:14] Check failed: H5LTfind_dataset(file_id, dataset_name_) Failed to find HDF5 dataset weight28
    *** Check failure stack trace: ***
        @     0x7f17077ec9fd  google::LogMessage::Fail()
        @     0x7f17077ee89d  google::LogMessage::SendToLog()
        @     0x7f17077ec5ec  google::LogMessage::Flush()
        @     0x7f17077ef1be  google::LogMessageFatal::~LogMessageFatal()
        @     0x7f1707e4d774  caffe::hdf5_load_nd_dataset_helper<>()
        @     0x7f1707e4bcf0  caffe::hdf5_load_nd_dataset<>()
        @     0x7f1707e8fd78  caffe::HDF5DataLayer<>::LoadHDF5FileData()
        @     0x7f1707e8ebf8  caffe::HDF5DataLayer<>::LayerSetUp()
        @     0x7f1707e283b2  caffe::Net<>::Init()
        @     0x7f1707e2ad85  caffe::Net<>::Net()
        @     0x7f1707e6da5f  caffe::Solver<>::InitTrainNet()
        @     0x7f1707e6df7b  caffe::Solver<>::Init()
        @     0x7f1707e6e3e8  caffe::Solver<>::Solver()
        @     0x7f1707e865a3  caffe::Creator_SGDSolver<>()
        @           0x4116b1  caffe::SolverRegistry<>::CreateSolver()
        @           0x40ac56  train()
        @           0x406e32  main
        @     0x7f17066adf45  (unknown)
        @           0x4074b6  (unknown)

I searched for this problem and it seems that my hdf5 file need a dataset label, but the fact is I don't need that. I only need a dataset of 91250x28x28. And feed it to the loss layer as weights. Here is my h5 file:

HDF5 weight28.h5 
Group '/' 
    Dataset 'data' 
        Size:  2555000x28
        MaxSize:  Infx28
        Datatype:   H5T_IEEE_F64LE (double)
        ChunkSize:  28x28
        Filters:  none
        FillValue:  0.000000

I modified the sigmiodcrossentropy layer to add it as a third bottom layer:

// modified here
  const Dtype* pixelweights = bottom[2]->cpu->data();

  Dtype loss = 0;
  for (int i = 0; i < count; ++i) {
    loss -= pixelweights(i)*(input_data[i] * (target[i] - (input_data[i] >= 0)) -
            log(1 + exp(input_data[i] - 2 * input_data[i] * (input_data[i] >= 0))));
  }
  top[0]->mutable_cpu_data()[0] = loss / num;
}

The proto file:

layer {
  name: "loss_G"
  type: "SigmoidCrossEntropyLoss"
  bottom: "global_smR"
  bottom: "mask28"
  bottom: "weight28"  //Added it here
  top: "loss_G"
}

I was expected that a batch of the data in the h5 file will be read to the net as bottom[2](size batchsize*28*28). Here is the question.

Can I get what I expected from the code above?
Do I have to add a label set in the h5 file to solve the error?
If I add it to the h5 file, how should I handle the extra label data in the loss layer?

Any advice will be appraciated, thanks!

score 1 · Answer 1 · answered Oct 15 '16 at 17:29

Your "HDF5Data" has top named "weight28", but your h5 file has only dataset "data". The "top" of "HDF5Data" layer must be the same as the Dataset name stored in the h5 file. If you have more than one dataset dtored in the same file, you can have multiple tops with the names of the Datasets in the h5 file.

caffe hdf5 H5LTfind_dataset(file_id, dataset_name_) Failed to find HDF5 dataset

1 Answers1