0

I am new to Caffe and appreciate it if someone guide me. I am going to make the lmdb from jpg using convert_imagenet.cpp, but cannot be succeeded. The content of the create.sh is as follows:

#!/usr/bin/env sh
# Create the caltech101 lmdb inputs
# N.B. set the path to the caltech101 train + val data dirs

EXAMPLE=examples/caltech101
DATA=data/caltech101
TOOLS=build/tools

TRAIN_DATA_ROOT=examples/caltech101/caltech101_train/
VAL_DATA_ROOT=examples/caltech101/caltech101_train/

# Set RESIZE=true to resize the images to 256x256. Leave as false if images have
# already been resized using another tool.
RESIZE=true
if $RESIZE; then
  RESIZE_HEIGHT=256
  RESIZE_WIDTH=256
else
  RESIZE_HEIGHT=0
  RESIZE_WIDTH=0
fi

if [ ! -d "$TRAIN_DATA_ROOT" ]; then
  echo "Error: TRAIN_DATA_ROOT is not a path to a directory: $TRAIN_DATA_ROOT"
  echo "Set the TRAIN_DATA_ROOT variable in create_caltech101.sh to the path" \
       "where the caltech101 training data is stored."
  exit 1
fi

if [ ! -d "$VAL_DATA_ROOT" ]; then
  echo "Error: VAL_DATA_ROOT is not a path to a directory: $VAL_DATA_ROOT"
  echo "Set the VAL_DATA_ROOT variable in create_caltech101.sh to the path" \
       "where the caltech101 validation data is stored."
  exit 1
fi

echo "Creating train lmdb..."

GLOG_logtostderr=1 $CAFFE_ROOT/build/tools/convert_imageset \
    --resize_height=256 \
    --resize_width=256 \
    --shuffle \
    $CAFFE_ROOT/examples/caltech101/caltech101_train/ \
    $CAFFE_ROOT/data/caltech101/caltech101_train.txt \
    $CAFFE_ROOT/examples/caltech101/caltech101_train_lmdb

echo "Creating val lmdb..."

GLOG_logtostderr=1 $TOOLS/convert_imageset \
    --resize_height=256 \
    --resize_width=256 \
    --shuffle \
    $CAFFE_ROOT/examples/caltech101/caltech101_train/ \
    $CAFFE_ROOT/data/caltech101/caltech101_test.txt \
    $CAFFE_ROOT/examples/caltech101/caltech101_test_lmdb

echo "Done."

Also, after executing the create.sh, the output is :

root@samar-Dell-Precision-M3800:~/caffe# ./examples/caltech101/create_caltech101.sh
Creating train lmdb...
I0701 20:17:34.589217  3256 convert_imageset.cpp:86] Shuffling data
I0701 20:17:34.589913  3256 convert_imageset.cpp:89] A total of 84 images.
I0701 20:17:34.590143  3256 db_lmdb.cpp:35] Opened lmdb /home/samar/caffe/examples/caltech101/caltech101_train_lmdb
E0701 20:17:34.645048  3256 io.cpp:80] Could not open or find file /home/samar/caffe/examples/caltech101/caltech101_train/
I0701 20:17:34.768301  3256 convert_imageset.cpp:153] Processed 83 files.
Creating val lmdb...
I0701 20:17:34.787917  3258 convert_imageset.cpp:86] Shuffling data
I0701 20:17:34.788311  3258 convert_imageset.cpp:89] A total of 40 images.
I0701 20:17:34.788477  3258 db_lmdb.cpp:35] Opened lmdb /home/samar/caffe/examples/caltech101/caltech101_test_lmdb
I0701 20:17:34.872882  3258 convert_imageset.cpp:153] Processed 40 files.
Done.

Two files including caltech101_train_lmdb and caltech101_test_lmdb are created but the size in unknown.

Thank you for your great help.

Nikolai Saiko
  • 1,679
  • 26
  • 27
user3549723
  • 1
  • 3

2 Answers2

1

First of all, use absolute paths for all the variables:

EXAMPLE
DATA
TOOLS

TRAIN_DATA_ROOT
VAL_DATA_ROOT

Next, you seem to have only 84 images. If that is the case, your DB will be really small. Check carefully if you have provided the training and validation images properly. You need to have two text files - one for training and another for validation. In each file you'll have two columns. In each row the first column contains the path to an image and the second column has the corresponding label. The label is always a number. The two columns are separated by a space.

Harsh Wardhan
  • 2,110
  • 10
  • 36
  • 51
  • Thank you for reply, I updated the above parameters as you mentioned and re-ran it, but the following error came up. I paste some part of the train.txt and test.txt and also create.sh file – user3549723 Jun 30 '16 at 07:05
  • EXAMPLE=/home/samar/caffe/examples/caltech101 DATA=/home/samar/caffe/data/caltech101/images TOOLS=/home/samar/caffe/build/tools TRAIN_DATA_ROOT=/home/samar/caffe/data/caltech101/images/ VAL_DATA_ROOT=/home/samar/caffe/data/caltech101/images/ – user3549723 Jun 30 '16 at 07:06
  • GLOG_logtostderr=1 $CAFFE_ROOT/build/tools/convert_imageset \ --resize_height=256 \ --resize_width=256 \ --shuffle \ $CAFFE_ROOT/data/caltech101/images/ \ $CAFFE_ROOT/data/caltech101/caltech101_train.txt \ $CAFFE_ROOT/data/caltech101/caltech101_train_lmdb #caltech101_train_lmdb doesn't exist. I think it will be created in this step echo "Creating val lmdb..." – user3549723 Jun 30 '16 at 07:07
  • GLOG_logtostderr=1 $TOOLS/convert_imageset \ --resize_height=256 \ --resize_width=256 \ --shuffle \ $CAFFE_ROOT/data/caltech101/images/ \ $CAFFE_ROOT/data/caltech101/caltech101_test.txt \ $CAFFE_ROOT/data/caltech101/caltech101_test_lmdb #caltech101_test_lmdb doesn't exist. I think it will be created in this step – user3549723 Jun 30 '16 at 07:07
  • cimage_0034.jpg 0 cimage_0035.jpg 0 cimage_0036.jpg 0 cimage_0037.jpg 0 image_0001.jpg 1 image_0002.jpg 1 image_0003.jpg 1 image_0004.jpg 1 image_0005.jpg 1 – user3549723 Jun 30 '16 at 07:08
0

for create database in lmbd:

create the train data folder create train.txt file conataining filename label create validation data folder create val.txt file contatining filename and label

*** edit this file examples/imagenet/create_imagenet.sh *************** :-)

EXAMPLE= path to where *.lmbd folder wil be stored
DATA= path where val.txt and train.txt is present
TOOLS=build/tools

TRAIN_DATA_ROOT=test/make_caffe_data/train/ # path to trainfiles
VAL_DATA_ROOT=test/make_caffe_data/val/ # path to test_files

Set RESIZE=true to resize the images to 256x256. Leave as false if images have

already been resized using another tool.

RESIZE=true  


./examples/imagenet/create_imagenet.sh
vishal
  • 855
  • 1
  • 8
  • 16