1

I'm trying to make a Training/Validation LMDB set for use with NVIDIA Digits, but I can't find any good examples/tutorials.

I understand how to create an LMDB database, but I'm uncertain on how to correctly format the data. I get how to create an image using the caffe_pb2 Datum by setting channels/width/height/data and save them.

But, how do I create the Labels LMDB? Do I still use a Caffe Datum? If so, what do I set the channels/width/height to? Will it work if I have a single value label?

Thanks

AdmiralJonB
  • 2,038
  • 3
  • 23
  • 27
  • have you looked at [`convert_imageset`](http://stackoverflow.com/questions/31427094/guide-to-use-convert-imageset-cpp/31431716#31431716)? – Shai Oct 25 '16 at 13:41
  • I haven't. Right now I'm using images; however, this is mainly for testing to see if things work. My actual work in the future won't be on images but on another feature vector, hence why I would like to understand how things work in preparation! – AdmiralJonB Oct 25 '16 at 13:44

1 Answers1

1

DIGITS only really supports data in LMDBs for now. Each value in the LMDB key/val store must be a Caffe Datum, which limits the number of dimensions to 3.

Even though Caffe Datums allow for a single numeric label (datum.label), when uploading a prebuilt LMDB to DIGITS you need to specify a separate database for the labels. That's inefficient if you only have a single numeric label (since you could have done it all in one DB), but it's more generic and scalable to other label types.

Sorry, you're right that this isn't documented very well right now. Here are some source files you can browse for inspiration if you're so inclined:

Luke Yeager
  • 1,400
  • 1
  • 17
  • 30
  • why not supporting HDF5 data? – Shai Oct 25 '16 at 20:03
  • 1
    Because (1) Caffe's HDF5 support is half-hearted at best and (2) it didn't take too much work to let Torch read from LMDBs. As we add more frameworks, supporting LMDB+caffe.datum for each framework makes less and less sense and we'll try to pick a format that each of the non-Caffe frameworks will use by default. – Luke Yeager Oct 25 '16 at 22:17