Connect to already running MATLAB with MATLAB Engine

Question

The MATLAB Engine API allows accessing MATLAB functionality from a C program. To set up a MATLAB Engine session, one calls the engOpen function. By default, this launches a new instance of MATLAB, for the exclusive use of Engine applications.

What I want is to use an already running MATLAB session from an Engine application so I can access its variables. I need to do this on Linux and OS X.

The Engine interface has very different implementations on Windows (where it uses a COM server) and Unix. On Unix we have to pass the location of the MATLAB executable to engOpen, while on Windows (where the Engine uses a COM server) we don't. On Windows it is possible to use an existing instance of MATLAB for Engine applications: just run enableservice('AutomationServer', true). Is there any solution on Unix systems?

I am hoping that there is a general solution—perhaps undocumented—because, based on the documentation, the Python interface seems to allow this. I tested this on OS X and it works. I can, for example, start up an interactive MATLAB session, set a variable a=5, then start Python, connect to the same session and be able to retrieve this variable. It is also possible to connect to an already running session from Java.

I need to do this from C however, not from Python, as it will be used in MATLink, the Mathematica-MATLink interface. How does the Python interface achieve this? How can I reproduce the same using C?

If there is an alternative C API than the "MATLAB Engine" that makes this possible (perhaps some other documented or undocumented C API that the Python interface is based on), I can accept that as an answer. I want a C program that can make use of an already running interactive MATLAB session.

---

Update:

Some poking around in the Python interface reveals that it makes use of a library called libmwengine_api. This is not the same as the documented MATLAB Engine C API. We can look at the symbols in this library. On OS X,

nm -g libmwengine_api.dylib | c++filt

Then we can google for these symbols or grep the MATLAB installation directory for files containing them. There is nothing in plain text that turns up.

Based on this I believe that the Python interface uses an undocumented C++ Engine API which is distinct from the old, documented C one.

Answer 1

"On the Mac and Linux® platforms, you cannot make an engine program connect to an existing MATLAB session."

Answer 2

Here is the source of its Python Implementation , see if it helps you.

Some Searching Gave Me The Impression that libmwengine is part of a library that is used in embedded system and which was long back obsoleted.

See This link , Library libmwengine_api was Obsoleted long back.Maybe That is why it is undocumented.

Answer 3

I am able to connect to an already-running MATLAB session in my command line on MacOS (should work on any Linux). I haven't figured out how to connect to the already-running MATLAB session from the MATLAB IDE, but my personal motivation was to not have to use the IDE. I keep a MATLAB kernel running in the background indefinitely and connect to it as I like. This allows my to run MATLAB scripts in batch mode from my text editor without having to spin up a new kernel each time.

Here's how I did it. If you want more details, I would be happy to provide the full scripts that accomplish it:

1. Start a MATLAB kernel/session from Python. Rather than the MATLAB Engine API, which I found hard to use, I opted for an existing iPython Jupyter Kernel interface called imatlab. I had to install this (pip install imatlab) as well as Jupyter (pip install jupyter):

from notebook.services.kernels.kernelmanager import MappingKernelManager
m = MappingKernelManager()
m.start_kernel(kernel_name="imatlab")
...

This creates a .json "connection file" in the working directory with the name kernel-kernel_id.json where kernel_id a UID (e.g. 4931ac70-e8fd-4d35-81b2-de53e07956c8). I also write the name of this kernel to a file in my Python script:

id = m.list_kernel_ids()[0]
text_file = open("/Users/Zach/.matlab_kernel/matlab_kernel_name.txt", "w")
text_file.write(id)
text_file.close()

I actually do this in a background TMUX session so I don't have to leave my terminal window open. Happy to comment on that as well.

2. Connect to this MATLAB kernel using Jupyter's console interface (though I could connect a Jupyter notebook connected to this kernel as well). First I have to retrieve the kernel_name from the file I wrote it to:

MATLABKERNELNAME=`cat /Users/Zach/.matlab_kernel/matlab_kernel_name.txt`
cd /Users/Zach/.matlab_kernel
jupyter console --existing $MATLABKERNELNAME

I also made a variant of this bash command that opens the kernel and sends the command to source a MATLAB .m file:

mtlb_existing_run_expect(){
        thefilename=`to_abs_path $1`
        thecurrentdirectory=`pwd`
        thekernelname=`cat /Users/Zach/.matlab_kernel/matlab_kernel_name.txt`
        expect <(cat <<EOF
            cd /Users/zach/.matlab_kernel
            spawn jupyter console --existing $thekernelname --simple-prompt --no-confirm-exit && exit
            expect ": "
            send "cd '$thecurrentdirectory'\n"
            expect ": "
            send "run('$thefilename')\n"
            interact -u "exp0"
EOF
)
}

You'll notice I always cd into the directory in which I started the kernel. That's because Jupyter creates that .json "connection file" there, and it automatically reads it if you're in the directory and give the kernel UID as the argument to the --existing flag.

I hope this is helpful to somebody!