2

I have a data set which form is matlab file.
The data set contains 600,000 samples and every sample is a matrix of 7-by-256.
My data is not image but signal.
I want to use CNN of caffe to train the data.
So how can I convert it to LMDB as my input of CNN.

I'm badly need the solution!

Shai
  • 111,146
  • 38
  • 238
  • 371
Zhao Wulanaren
  • 161
  • 1
  • 2
  • 9

1 Answers1

2

Converting data in matlab directly to lmdb might be a little tricky.

Why don't you try exporting your data to hdf5 binary files (supported both by matlab and caffe)?

Here is an answer describing how this can be done.

Community
  • 1
  • 1
Shai
  • 111,146
  • 38
  • 238
  • 371
  • As I mentioned above,my data set contains 600,000 samples and every sample is a matrix of 7-by-256 and my data is not image but signal. So does it means the blob is my case is [600000,1792,1,1]?And actually I don't know clearly what [num,channel,height,width] of blob stands for especially the height and width.Can you explain it for me?@Shai – Zhao Wulanaren Apr 05 '16 at 07:35
  • @ZhaoWulanaren the terms are for images: channels are usually color RGB and there is the width and height of the image. In your case I'm not sure what dimmensions 7 and 256 stands for, but you can treat it as a 2D signal with channel=1 height=7 and width=256 – Shai Apr 05 '16 at 07:39
  • My row data is 1-D and that is to say every row sample is 1-by 256.I use wave transform with 6 order to get the detail coefficient of every order and the appropriate coefficient of the last order.So every sample is changed to a matrix of 7-by-256. – Zhao Wulanaren Apr 05 '16 at 07:57
  • @ZhaoWulanaren then it's up to you to decide if you feed the net with the original 1D signal or the 2D transformed one – Shai Apr 05 '16 at 07:59
  • My row data is 1-D and that is to say every row sample is 1-by 256.I use wave transform with 6 order to get the detail coefficient of every order and the appropriate coefficient of the last order.So every sample is changed to a matrix of 7-by-256.You said the height=7 and width=256 in my case .So can I use caffe model of image recognition with my data set? – Zhao Wulanaren Apr 05 '16 at 08:11
  • @ZhaoWulanaren give it a try – Shai Apr 05 '16 at 08:17
  • OK,but I know height is equal to width in image recognition and kernel size (or pooling size)is square matrices.So should I edit the kernel size(or pooling size) of the model to fit my non-square matrix data set? – Zhao Wulanaren Apr 05 '16 at 08:24
  • @ZhaoWulanaren you can change them. but this is WAY too broad form comments to discuss this – Shai Apr 05 '16 at 08:29
  • Thank you for your advice! I''ll try it first. – Zhao Wulanaren Apr 05 '16 at 08:34
  • I store my data as 3-D matrix that is 7*256*600000. So if I want to get the blob,should I do as the following way: reshape(data,[600000,1,7,256])@Shai – Zhao Wulanaren Apr 05 '16 at 11:41
  • please see [this post](http://stackoverflow.com/a/34431532/1714410) regarding the "shape" of blobs in matlab. But you need a 256x7x1x600000 array in matlab. and do **NOT** use `reshape` to change the order of dimensions, use `permute` for that. if you confuse `reshape` with `premute` you turn your data into garbage. – Shai Apr 05 '16 at 11:53
  • Thank you for your help! – Zhao Wulanaren Apr 05 '16 at 14:02
  • @ Shai Hello ,now I get the data blob of HDF5. Then I should do mean subtraction but the function 'compute_image_mean.cpp' in Caffe is not fit for HDF5.So how to do it? – Zhao Wulanaren Apr 08 '16 at 01:35
  • @ZhaoWulanaren I suppose that's a nice question. why don't you post it as such? – Shai Apr 08 '16 at 03:53