2

I'm trying to train a CNN on my own dataset using Caffe framework, and it is highly recommended that the dataset be converted to the lmdb or leveldb formats due to speed efficiency. To do so, all images must be put into a single folder and the 'list.txt' must be prepared accordingly. My own dataset is so huge and in so many folders and subfolders so that it would be so laborious copy all of them into a single folder. Thereforeو I'm wondering to know whether there exist any alternative way to generate the lmdb file without need to copy all images into a single folder.

Shai
  • 111,146
  • 38
  • 238
  • 371
Saeed
  • 742
  • 1
  • 7
  • 21

2 Answers2

1

There are (at least) two solutions to your problem.

  1. Don't copy the files to a single folder, just create symbolic links.

  2. All the images do not have to be in the same folder. You can have full paths in 'list.txt' file. For example:

/path/to/image.jpg 0
/another/path/class01.jpg 1
/yet/another/path/class0.jpg 0

And so on...

Shai
  • 111,146
  • 38
  • 238
  • 371
  • Thanks for your help. So, another arising question is if we put the full dir in the list.txt, what we should pass as "/path/to/jpegs/" argument into the convert_imageset tool. See the following link http://stackoverflow.com/questions/31427094/guide-to-use-convert-imageset-cpp – Saeed Aug 30 '15 at 10:17
  • @user3062492 try passing the root folder `/` as argument – Shai Aug 30 '15 at 10:26
  • Sure, I'll check it out. Thanks – Saeed Aug 30 '15 at 13:50
  • @Shai, if you use root foder argument with "/" and `list.txt` file contains absolute path for image when calling `convert_imageset`, then resulting path used in `convert_imageset` would be `//path/to/image.jpg 0`. It's incorrect path in linux. The path for image in `list.txt` should be relative path to `/` for correct operation of `convert_imageset`. – Hongsoog Sep 13 '20 at 07:31
1

For second solution suggested by @Shai

If you use root foder argument with / and list.txt file contains absolute path for image when calling convert_imageset, then resulting path used in convert_imageset would start with double slash (ex. //path/to/image.jpg). It's incorrect path for corresponding image file in linux.

The path for image in list.txt should contains relative path to / for correct operation of convert_imageset as follows:

path/to/image.jpg 0
another/path/class01.jpg 1
yet/another/path/class0.jpg 0

instead of

/path/to/image.jpg 0
/another/path/class01.jpg 1
/yet/another/path/class0.jpg 0
Hongsoog
  • 543
  • 4
  • 9