1

I have the following structure in a .txt file:

/path/to/image x y
/path/to/image x y

where x and y are integers.

What I want to do now is: Create a hdf5 file to use in Caffe ('train.prototxt')

My Python code looks like this:

import h5py
import numpy as np
import os

text = 'train'
text_dir = text + '.txt'

data = np.genfromtxt(text_dir, delimiter=" ", dtype=None)

h = h5py.File(text + '.hdf5', 'w')
h.create_dataset('data', data=data[:1])
h.create_dataset('label', data=data[1:])

with open(text + "_hdf5.txt", "w") as textfile:
    textfile.write(os.getcwd() + '/' +text + '.hdf5')

But this does not work! Any ideas what could be wrong?

1 Answers1

1

It does not work because your 'data' is /path/to/image instead of the image itself.

See this answer for more information.

Graham
  • 7,431
  • 18
  • 59
  • 84
Shai
  • 111,146
  • 38
  • 238
  • 371
  • Yeah I have looked at that answer. But that answer is related to single labels: **y = np.zeros( (1,len(lines)), dtype='f4' )** **y[i] = float(sp[1])** How would you create that for multi label classification? –  Oct 25 '16 at 12:17
  • you should have `y` with `shape` "num-of-examples"-by-"dim of y" – Shai Oct 25 '16 at 12:23
  • I have two classes and each class has 36 labels. You mean num-exaxmples = num images and dim of y is 2? y = np.zeros( (2,num-images), dtype='f4' ) –  Oct 25 '16 at 12:25
  • @I'd try ` y = np.zeros( (num_images,2), dtype='f4' )` – Shai Oct 25 '16 at 12:27
  • Yeah exactly! this it what I have in my new implementation. And then you do y[i][0] = float(sp[1]) and y[i][1] = float(sp[2])? The next question would be how to use it in .prototxt regarding the loss function do you want me to upload my .prototxt? –  Oct 25 '16 at 12:29
  • more like `y[i,0] = float(sp[1])` `y[i,1] = float(sp[2])`, I suppose. – Shai Oct 25 '16 at 12:31
  • Ok that means that num_output is 2 in my fc layer? –  Oct 25 '16 at 12:35
  • @thigi, no. That means you need 2 fc layers with `num_output` is 36 (you have 36 labels). I suppose it make more sense to have `y1` and `y2` with 1d label, rather than `y` with 2d label... – Shai Oct 25 '16 at 12:40
  • Ok thanks, let me update it! I have loads of fc layers but I only need two "last fc" layers? –  Oct 25 '16 at 12:42
  • @thigi you need a classification layer ("last fc") for predicting each label. In your case you have two labels (with 36 classes) than you should have two "last fc" with `num_output: 36`. – Shai Oct 25 '16 at 12:43
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/126622/discussion-between-thigi-and-shai). –  Oct 25 '16 at 12:44
  • @thigi your current question is quite different than the original one you posted. Please revert to the original question and feel free to ask new ones. You might find linking new questions to old once useful for context. – Shai Oct 25 '16 at 13:00
  • Done: http://stackoverflow.com/questions/40240756/caffe-two-class-multi-label-classification-with-hdf5 Here is the new question. Could you help me with this one? Thank you!! –  Oct 25 '16 at 13:03
  • When trying to solve the problem I get this error when I have y1 and y2 Check failed: shape[i] <= 0x7fffffff / count_ (256 vs. 242) blob size exceeds INT_MAX –  Oct 25 '16 at 13:07
  • you need to ask question as questions and not as comments. It is difficult to follow, and you will never get answers for other users. – Shai Oct 25 '16 at 13:15