2

Does anybody know how to create a C# client for tensorflow serving?

My tensorflow serving installation:

I installed tensorflow serving using the tensorflow serving dockerfile, then inside the container I did the following:

pip install tensorflow

pip install tensorflow-serving-api

echo "deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | tee /etc/apt/sources.list.d/tensorflow-serving.list

curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | apt-key add -
apt-get update && apt-get install tensorflow-model-server

Then I run the tensorflow serving server:

tensorflow_model_server --port=9000 --model_name=example_model --model_base_path=/serving/my_model_2 &> my_log &

where my_model_2 contains the exported tensorflow model I want to serve.

Given this information I have the following questions:

  • Do I need to install tensorflow serving in a different way in order to make a C# client? If I need to install it in a different way; Can you tell me how?
  • Can you give me a general idea of what I have to do in order to accomplish my goal? I mean, I suspect I have to install my tf serving in a different way in order to create a explicit .proto file . I'm a little lost about this if you can give me the general idea and an example I would appreciate it.
Rand Random
  • 7,300
  • 10
  • 40
  • 88
Diego Orellana
  • 994
  • 1
  • 9
  • 20

1 Answers1

3

As far as I understand, you need the proto files to generate a tensorflow serving client in C# for the grpc services.

https://github.com/Wertugo/TensorFlowServingCSharpClient This is one example I am following. Its the same MNIST example with C# client.

Hope this helps.

Please update here if you have got any better options.

RTS
  • 56
  • 7
  • Could you please share the c# client if possible? I am still trying to figure out how to implement the c# client. Any help is appreciated. – RTS Apr 10 '18 at 18:50
  • How did you convert the proto files into .cs files? I tried packages\Grpc.Tools.1.10.x\tools\windows_x86\protoc.exe -I/Docments/files/tensorflow_serving/apis --csharp_out services --grpc_out services /Documents/tensorflow_serving/apis/*.proto --plugin=protoc-gen-grpc=packages/Grpc.Tools.1.10.x/tools/windows_x86/grpc_csharp_plugin.exe Is this the way to convert the .proto files to .cs? – RTS Apr 10 '18 at 20:10
  • 1
    You don't need to compile them, @Wertugo already did it, they are in the Service folder(https://github.com/Wertugo/TensorFlowServingCSharpClient/tree/master/src/BaseLibs/TensorFlowServingClient/Service). You just need to use them. – Diego Orellana Apr 11 '18 at 01:43
  • In the folder, I mentioned above you find all the classes necessary to communicate with TensorFlow serving from C#. You can compile them again but is not necessary so I didn't do it. Thus, I don't have the exact command. – Diego Orellana Apr 11 '18 at 01:46
  • Thankyou! Is it that you just add the service folder to the new project? Did you add the utils and properties folders as well? – RTS Apr 11 '18 at 11:39
  • 1
    The util folder is to create the message with the image data. If you want to implement a model that does not manipulate images then you don't need the Util's classes, (he created those classes, he did not create the services classes (they are the product of compiling the proto files)) but you do need all the Service's classes. You don't need the properties. Give me points! Sorry for the late answer. – Diego Orellana Apr 15 '18 at 05:33
  • THankyou. figured that out. But now I have another error. Input and filter must have same depth:1Vs 3 – RTS Apr 16 '18 at 15:25
  • are you trying to run the MNIST or a custom model? – Diego Orellana Apr 17 '18 at 22:33
  • @DiegoOrellana Do you have any thoughts on the error : Input and filter must have same depth:1Vs 3 – RTS May 01 '18 at 18:40
  • @RTS can you please have a look here https://stackoverflow.com/questions/65869272/raggedtensor-request-to-tensorflow-serving-fails ? – Shlomi Schwartz Jan 26 '21 at 08:41