1

I started with Caffe and the mnist example ran well.
I have the train and label data as data.mat. (I have 300 training data with 30 features and labels are (-1, +1) that have saved in data.mat).

However, I don't quite understand how I can use caffe to implement my own dataset?

Is there a step by step tutorial can teach me?

Many thanks!!!! Any advice would be appreciated!

Shai
  • 111,146
  • 38
  • 238
  • 371
ahmad navid ghanizadeh
  • 139
  • 9

1 Answers1

2

I think the most straight forward way to transfer data from Matlab to caffe is via HDF5 file.

First, save your data in Matlab in an HDF5 file using hdf5write. I assume your training data is stored in a variable name X of size 300-by-30 and the labels are stored in y a 300-by-1 vector:

hdf5write('my_data.h5', '/X', 
  single( permute(reshape(X,[300, 30, 1, 1]),[4:-1:1]) ) );
hdf5write('my_data.h5', '/label', 
  single( permute(reshape(y,[300, 1, 1, 1]),[4:-1:1]) ), 
  'WriteMode', 'append' );

Note that the data is saved as a 4D array: the first dimension is the number of features, second one is the feature's dimension and the last two are 1 (representing no spatial dimensions). Also note that the names given to the data in the HDF5 are "X" and "label" - these names should be used as the "top" blobs of the input data layer.

Why permute? please see this answer for an explanation.

You also need to prepare a text file listing the names of all hdf5 files you are using (in your case, only my_data.h5). File /path/to/list/file.txt should have a single line

/path/to/my_data.h5

Now you can add an input data layer to your train_val.prototxt

layer {
  type: "HDF5Data"
  name: "data"
  top: "X"     # note: same name as in HDF5
  top: "label" # 
  hdf5_data_param {
    source: "/path/to/list/file.txt"
    batch_size: 20
  }
  include { phase: TRAIN }
}

For more information regarding hdf5 input layer, you can see in this answer.

Community
  • 1
  • 1
Shai
  • 111,146
  • 38
  • 238
  • 371
  • Did you manage to run training of a net on MATLAB using Caffe? – Royi Aug 18 '16 at 19:53
  • @Drazick sorry but I do not mix Matlab and caffe. The Python interface is much better – Shai Aug 18 '16 at 20:01
  • I saw you answered a lot of troubled MATLAB users hence I thought you have some experience. It really seems the MATLAB interface is buggy. I have something working on Command Line yet crashes MATLAB. – Royi Aug 18 '16 at 20:03
  • @Drazick I used to work extensively with Matlab, but nowadays I am more of a Python coder... – Shai Aug 18 '16 at 20:15
  • Do you have any experience with Caffe on MATLAB? Thank You. – Royi Aug 18 '16 at 21:24
  • @Drazick sorry, no. But I strongly recommend you make the effort to use the Python interface – Shai Aug 18 '16 at 21:25
  • Well, the problem is they don't support Python 3.5 on Windows. – Royi Aug 19 '16 at 06:49