Evade polling when accessing Matlab from Python

Question

I would like to access Matlab from Python (on Windows, remotely and via COM-interface). My goal is: Matlab is doing some work and permanently changes the value of a certain variable. I need to know when the value exceeds some constant. Right now, I'm polling Matlab for the value of that variable in an indefinite loop which breaks on exceeding the value. However, I would like to let Matlab do the work and tell me when that is the case, while I'm lazily sitting there listening. Is there a way to achieve this, and how is it done best? I have thought of defining a callback function I'm passing to Matlab, which on the exceed-event triggers a break out of a non-busy-wait-loop in Python, but I doubt it would work. I'm not very experienced neither in Matlab nor Python, so cues are much appreciated.

There's a lot of other code involved, but basically right now it is like

connectToMatlab(*args)
while True:
    val = getValueFromMatlab()
    if val > constant or timeout: break

What I have in mind is

def breakLoop():
    ...  

connectToMatlab(breakLoop, *args)
while True:
    time.sleep(1) # or some alternate non-busy-wait

and then let Matlab invoke breakLoop() upon val > constant. However, I don't know if it is possible to let Matlab do that with a callback and if yes, how to implement such a breakLoop()-Function.

Answer 1

You can go about this the other way, and use the file system as a way to pass the messages between MATLAB and Python.

Inside your MATLAB code, each time you change the variable, check if it exceed some threshold. If it does, create a new file in a predetermined location. Think of this as triggering an event.

Now inside your python code, use some of the available ways to the listen to changes in the file system, and respond by indicating some variable to break the loop.

---

EDIT

Here is a skeleton of the solution proposed:

matlab_script.m

%# directory that Python code is watching for modifications
dirPath = 'some_directory';

x = 0;
for i=1:1000
    %# some lengthy operation
    pause(0.5)
    x = x + 1;

    %# check if variable exceeds threshold
    if x > 10
        %# save the workspace to MAT-file inside the directory watched.
        %# this shall trigger the notification in Python
        save( fullfile(dirPath,'out.mat') )
        break
    end
end

python_code.py

import os, sys, time
import win32file, win32event, win32con

# stub your functions in my case
def connectToMatlab():
  pass
def getValueFromMatlab():
  return 99

# path to predetermined directory to watch
dirPath = "some_directory"
dirPath = os.path.abspath(dirPath)

# start/connect to a MATLAB session, running the script above
connectToMatlab()

# set up folder watching (notify on file addition/deletion/renaming)
print "Started watching '%s' at %s" % (dirPath, time.asctime())
change_handle = win32file.FindFirstChangeNotification(
  dirPath, 0, win32con.FILE_NOTIFY_CHANGE_FILE_NAME)

# time-out in 10 sec (win32event.INFINITE to wait indefinitely)
timeout = 10000

try:
  # block/wait for notification
  result = win32event.WaitForSingleObject(change_handle, timeout)

  # returned because of a change notification
  if result == win32con.WAIT_OBJECT_0:
    # retrieve final result from MATLAB
    print "MALTAB variable has exceeded threshold at %s" % time.asctime()
    val = getValueFromMatlab()

  # timed out
  elif result == win32con.WAIT_TIMEOUT:
    print "timed-out after %s msec at %s" % (timeout,time.asctime())
    val = None    # maybe to indicate failure

finally:
  # cleanup properly
  win32file.FindCloseChangeNotification(change_handle)

# work with val
print val

The WaitForSingleObject function start by checking the state of the specified object. If it is nonsignaled, the calling thread enters an efficient wait state and consumes very little processor time while waiting until the object is signaled (or the time-out interval elapses).

You see when the thread references the object which is in non-signaled state, there is an immediate context switch i.e. it is taken down from the processor and put into a wait/sleep mode. Later when the object is signaled, the thread is put back to the runnable queue and is ready for execution.

In this kind of waiting there is no wastage of CPU cycles in wait state, although there is a some overhead in context switching.

Compare this against the "poll-and-wait" approach, where the thread waits and checks for the state of the object of interest in some kind of a loop. This is known as spin or busy wait, which can prove to be a wastage of CPU cycles.

Now thanks to pywin32 module, we can use use those WaitFor... functions directly. The implementation should be a straightforward port of the standard example given in MSDN.

Alternatively you can use the PyQt library with its QFileSystemWatcher class instead of directly using the Win32 API.