2

I'm developing a simple game for a university project using Unity. This game makes use of machine learning, so I need TensorFlow in order to build a Neural Network (NN) to accomplish certain actions in the game depending on the prediction of the NN.

In particular my learning approach is reinforcement learning. I need to monitor the states and the rewards in the environment [coded in C#], and pass them to the NN [coded in Python]. Then the prediction [from Python code] should be sent back to the environment [to C# code].

Sadly I'm quite confused on how to let C# and Python communicate. I'm reading a lot online but nothing really helped me. Can anybody clear my ideas? Thank you.

Matt
  • 773
  • 2
  • 15
  • 32
  • Which OS are you using? You can just run Python scripts from C# as system commands. – Emmet B Feb 27 '18 at 20:14
  • @EmmetB thanks for this answer, I'm using MacOS. Ok, I understand so I may run a Python script from C# code sending informations to my NN. But I don't have any idea on how to transfer the prediction made, back to my C# code, for now, but that's another story. I may look for how to do it. Thanks. – Matt Feb 27 '18 at 20:18
  • 1
    https://stackoverflow.com/questions/206323/how-to-execute-command-line-in-c-get-std-out-results – Emmet B Feb 27 '18 at 20:20
  • If you're allowed to, you may use the UnityML library. It's pretty easy to set up. – Brandon Miller Feb 27 '18 at 21:04

6 Answers6

3

On thing for inter-process communication that is definitely worth looking at is (Win32) named pipes. It is almost as easy as writing to or reading from files. Easier than TCP/IP and you can even communicate over the local network, at least if you have the access rights. Add your own serialization code and your done.

It is a little asymmetric though (in one direction you just open a file with the name of the pipe, in the opposite direction you have to create a named pipe object, don't remember which is which).

For example see Example of Named Pipes

If you google for "named pipes windows" you will find loads of information. But remember not to use anonymous pipes (they are ugly).

oliver
  • 2,771
  • 15
  • 32
  • That's a very useful solution I omitted in purpose from my answer. From my experience, many just find it difficult, for some reason. But it's definitely worth a shot as it's (theoretically) simpler than my solutions. – Yotam Salmon Feb 27 '18 at 20:27
  • 1
    Yeah, I remember that I had to do some research back then, but it's worth it in my opinion. – oliver Feb 27 '18 at 20:29
  • Which method has the least overhead : UDP/TCP connection or Named pipes? – Karan Joisher May 11 '18 at 07:29
  • 1
    @KaranJoisher: If you are using pipes over the local network then they are transported by tcp, so they definitely have higher overhead than using tcp directly. On the same machine they seem to be about the same. See https://stackoverflow.com/questions/10872557/how-slow-are-tcp-sockets-compared-to-named-pipes-on-windows-for-localhost-ipc – oliver May 15 '18 at 16:28
  • So if on the same machine TCP and named pipes are giving similar performance then between TCP and UDP which would be better? Cause data transmission for my appication needs to be real time so it seems UDP should be the preferred choice – Karan Joisher May 24 '18 at 06:43
  • @KaranJoisher: I'm afraid I don't know. If I were you, I would just test it with some simple example. – oliver May 24 '18 at 13:52
2

You have a few options:

Subprocess

You can open the python script via the Unity's C# then send stdout and stdin data to and from the process. In the Python side it's as simple as input() and print(), and in the C# side it's basically reading and writing from a Stream object (as far as I remember)

UDP/TCP sockets

You can make your Python a UDP/TCP server (preferrably UDP if you have to transfer a lot of data, and it might be simpler to code). Then you create a C# client and send requests to the Python server. The python server will do the processing (AI magic, yayy!) then return the results to the Unity's C#. In C# you'd have to research the UdpClient class, and in Python, the socket module.

Yotam Salmon
  • 2,400
  • 22
  • 36
2

You can integrate the compiled DLL from C# natively with python using Python for .NET (pythonnet)

pythonnet is a package that gives Python programmers nearly seamless integration with the .NET 4.0+ Common Language Runtime (CLR) on Windows and Mono runtime on Linux and OSX.

Using this package you can script .NET applications or build entire applications in Python, using .NET services and components written in any language that targets the CLR (C#, VB.NET, F#, C++/CLI).

Installation

    pip install pythonnet

using compiled class library My.dll in python:

    import clr
    clr.AddReference("My.DLL")
    # call methods in dll

See a working example

pythonnet in github

How to use the package in python WIKI

M.Hassan
  • 10,282
  • 5
  • 65
  • 84
0

If you are using Unity Machine learning agent, then connecting C# code and python is not that hard.

The mlagents Python package contains two components: a low-level API which allows you to interact directly with a Unity Environment (mlagents_envs) and an entry point to train (mlagents-learn) which allows you to train agents in Unity Environments using our implementations of reinforcement learning or imitation learning. Follow this documentation for more details https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Training-ML-Agents.md

Gunjan Paul
  • 511
  • 6
  • 12
0

There are several ways to communicate between C# and Python. Here are some of the most common approaches:

Using a pipe: You can use a pipe to connect the C# application to the Python script. The Python script should be looking at standard input using readline to get the input. The C# should send output to the Python script standard input and make sure there is a linefeed at the end of each command 1.

Using IronPython: IronPython is an open-source implementation of the Python programming language that is tightly integrated with the .NET Framework. You can use IronPython to write Python code that can be called from C# 2.

Using Pyrolite: Pyrolite is a library that allows you to call Python code from C#. It works by running your Python code as a Pyro4 server 2.

Using TCP/IP: You can use TCP/IP sockets to communicate between C# and Python. This approach requires you to write code on both sides of the communication channel 3.

The best approach for your machine learning task will depend on your specific requirements and constraints. I hope this helps! Let me know if you have any other questions.

Hailu Worku
  • 14
  • 1
-1

Run with ML.NET C# code a TensorFlow model exported from Azure Cognitive Services Custom Vision. for more information

Hailu Worku
  • 14
  • 1
  • Hello Hailu, Stack Overflow recommends against providing an answer via a hyperlink, since if the link gets broken, the answer has no more value. Please try and add some more explanation to your answer about how it solves the problem. – Luke Briner Mar 03 '22 at 16:01