4

I open this thread to discuss how to bring my NN model to deployment.

I build and trained a NN in Matlab with mdCNN, (mdCNN is a simple Matlab library for building NN for multiple dimension input, which is currently is not supported with Matlab - cov3x3x3). I trained my model in Matlab, Now I want to bring it to production.

After few hours of research, I plan to do the following

  1. Train a NN model in Keras with TF backend. I choose Keras because I want to have backward compatibility with Matlab in the future.

  2. Grab a tensorflow session from Keras model, there is an example how to do that here. Than Save the session in *.pd file

  3. Load the NN model from openCV dnn model. there is a specific function that does that

    cv::readNet()

  4. Run the NN in C++ using OpenCV with 

    net.setInput(blob);
Mat prob = net.forward();

I want to check with you if this flow would really work. Are there any suggestions how to do the deployment better? Any suggestions or improvements for the flow ?

halfer
  • 19,824
  • 17
  • 99
  • 186
TripleS
  • 1,216
  • 1
  • 23
  • 39
  • Have you looked into [Tensorflow Lite](https://www.tensorflow.org/mobile/tflite/)? I don't know if your ultimate deployment is embedded (raspi, ios, andorid) but TFlite is written in c++. – James Poag Aug 28 '18 at 10:02
  • Thank you, I don't think that's what I need, Tensorflow lite is intended for mobile devices – TripleS Aug 28 '18 at 10:13
  • 1
    your flow should work, but you'll have to make sure that the preprocessing of new images will be numerically the same you've used during training. I suggest to write some unit tests and network/input examples to prove it. – Micka Aug 28 '18 at 10:27

1 Answers1

0

Maybe have a look at this question: Convert Keras model to C++

The general idea is to save the model in json and the weights in hdf5 and use this keras2cpp solution to convert it to C++.

WurzelseppQX
  • 520
  • 1
  • 6
  • 17
  • I already looked at it, the question is how to from a json file to a c++ framework – TripleS Aug 28 '18 at 09:52
  • Have a look at the `keras2cpp` link. You can find an example there how to use it . You just need to have python installed on your machine, then you can just use: `python dump_to_simple_cpp.py -a pathto/yourmodel.json -w pathto/yourweights.h5 -o pathto/youroutputfile.nnet`. – WurzelseppQX Aug 28 '18 at 12:55
  • This project is definitely not the way to do it, this project was last updated last year and support simple CNN and limited in activation layers. "The code is prepared to support simple Convolutional network (from MNIST example) but can be easily extended. There are implemented only ReLU and Softmax activations." I think that it's better to use a more solid c++ framework, like opencv, tensorflow c++ or Intel clDNN – TripleS Aug 28 '18 at 13:39