Modifying Caffe to accept 16 bit data inside lmdb

Question

I'm trying to make some modifications on Caffe in order to accept my 16 bit data.
I succeded into creating a lmdb dataset filled with 16 bit unsigned, unencoded 256x256 image instead of usual caffe 8 bit unsigned data, saved as "string" as would be the usual 8 bit lmdb that can be created with image_convert or DIGITS utilities.
I've changed the io.py functions array_to_datum and datum_to_array in order to create this lmdb at 16bit "string" data. Now, if I use this lmdb on caffe (just 4 classes), the networks runs but doesn't converge. I strongly suspect that is not reading my data properly. Problem is, the io.py functions array_to_blobproto and blobproto_to_array doesn't seems to make any distinction between internal data contents, and I cannot find the code where I should modify for dealing with 16bits.
Could anyone give me an hint to where to work on?

Edit: Messing around in the code, I think that one of the possibility should be to create a new data layer or a new image data layer if I would like to work directly on the png without going throught lmdb. But trying to modify that C++ code is not a trivial task for me, esp. I cannot easily follow the data flow inside the code. I see that new layer can be written in python. Do you think that a new input data layer could work nicely or would slow down the cnn performance?

Answer 1

I don't know much about converting and adapting caffe/lmdb interface, but it seems like a very risky path to take if you are not 100% certain with what you are doing.
For instance, you changed the io functions in the python interface, but I don't think caffe is using this interface when running from command line (e.g., $CAFFE_ROOT/build/tools/caffe train ...). Have you looked into the cpp io functions in io.cpp file?

I would strongly suggest an alternative path: use hdf5 binary inputs instead of lmdb.
You can convert the 16bit images you have to float32 and store them in hdf5 files, then input them to caffe via "HDF5Data" layer.

Read more about hdf5 and caffe in this thread.