1

I want to read data from hdf5 file and do data augmentation.

I can do data augmentation in transform_param with mirror and crop.

However,I cannot have transformation_param in a HDF5Data layer - caffe does not support this.

I consider it might be a good way to write a python layer to solve it. Read data from hdf5 file and do data augmentation at the same time.

I have read some relevant answers such as "caffe data layer example step by step" and "What is a Python layer in caffe?"

I still don't know some details such as where should I put the written file"*.py"? And how to write the codes to do data augmentation?

Community
  • 1
  • 1
egg
  • 11
  • 1

1 Answers1

0

You python layer *.py should be in your $PYTHONPATH.

Consider examples of python layers in $CAFFE_ROOT/examples/pycaffe/layers ,for custom python loss and data layer.

Consider particularly in $CAFFE_ROOT/examples/pycaffe/layers/pascal_multilabel_datalayers.py

Before next batch gets loaded, you can do the data augmentation on the fly.

Then in *.prototxt

layer {
  name: "data"
  type: "Python"
  top: "data"
  top: "label"
  python_param {
    module: "pascalcontext_layers"
    layer: "PASCALContextSegDataLayer"
    param_str: "{\'context_dir\': \'../../data/pascal-context\', \'seed\': 1337, \'split\': \'train\', \'voc_dir\': \'../../data/pascal\'}"
  }
}

module is the file name and layer is the class name.

curio17
  • 660
  • 1
  • 6
  • 15
  • Thanks for answering! I've tried this and found 'Check failed: registry.count(type) == 1 (0 vs. 1) Unknown layer type: Python'. However, I 'list(caffe.layer_type_list())' and find 'python' in this list. Why? – egg Feb 16 '17 at 11:37
  • You have to compile your caffe with `WITH_PYTHON_LAYER := 1`. See http://stackoverflow.com/questions/41344168/what-is-a-python-layer-in-caffe?noredirect=1&lq=1 for details – curio17 Feb 16 '17 at 11:40
  • I've compile that. I find 'python' when I run `list(caffe.layer_type_list())`. Maybe caffe just can't find it? – egg Feb 20 '17 at 01:49