I'd like to use http://www.losonczylab.org/sima/1.2/ from within MATLAB.
I can run it fine from python.
I can import example dependencies in MATLAB.
$ /opt/local/bin/python
Python 2.7.9 (default, Dec 13 2014, 15:13:49)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from sima import sequence
>>> sequence
<module 'sima.sequence' from '/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/sequence.pyc'>
>>> from sima import ImagingDataset
>>> ImagingDataset
<class 'sima.imaging.ImagingDataset'>
in MATLAB:
>> pyversion /opt/local/bin/python
>> pyversion
version: '2.7'
executable: '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Cont...'
library: '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib'
home: '/opt/local/Library/Frameworks/Python.framework/Versions/2.7'
isloaded: 0
>> import py.picos.tools.available_solvers
>> available_solvers
ans =
Python list with no properties.
['cvxopt', 'mosek7']
>> import py.numpy.empty
>> empty(0)
ans =
Python ndarray with properties:
T: [1x1 py.numpy.ndarray]
base: [1x1 py.NoneType]
ctypes: [1x1 py.numpy.core._internal._ctypes]
data: [1x0 py.buffer]
dtype: [1x1 py.numpy.dtype]
flags: [1x1 py.numpy.flagsobj]
flat: [1x1 py.numpy.flatiter]
imag: [1x1 py.numpy.ndarray]
itemsize: 8
nbytes: 0
ndim: 1
real: [1x1 py.numpy.ndarray]
shape: [1x1 py.tuple]
size: 0
strides: [1x1 py.tuple]
[]
>> import py.sima.sequence
Error using import
Import argument 'py.sima.sequence' cannot be found or cannot be imported.
>> import py.sima.Sequence
Error using import
Import argument 'py.sima.Sequence' cannot be found or cannot be imported.
>> import py.sima.Sequence.create
Error using import
Import argument 'py.sima.Sequence.create' cannot be found or cannot be imported.
>> import py.sima.sequence.create
Error using import
Import argument 'py.sima.sequence.create' cannot be found or cannot be imported.
>> import py.sima.sequence
Error using import
Import argument 'py.sima.sequence' cannot be found or cannot be imported.
>> import py.sima.Sequence
Error using import
Import argument 'py.sima.Sequence' cannot be found or cannot be imported.
>> import py.sima.ImagingDataset
Error using import
Import argument 'py.sima.ImagingDataset' cannot be found or cannot be imported.
>> import py.sima.imagingdataset
Error using import
Import argument 'py.sima.imagingdataset' cannot be found or cannot be imported.
>> import py.sima
Error using import
Import argument 'py.sima' cannot be found or cannot be imported.
How can I figure out why sima
can't be imported?
sima
is on the path:
>> pp = py.pprint.PrettyPrinter
pp =
Python PrettyPrinter with no properties.
<pprint.PrettyPrinter instance at 0x129425bd8>
>> pp.pprint(py.sys.path)
['',
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/readline',
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
'/Users/eflister/Library/Python/2.7/lib/python/site-packages',
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages']
I came up with the following tedious method to find the problem imports -- is there a better way?
- define a module as in http://www.mathworks.com/help/matlab/matlab_external/call-user-defined-custom-module.html
- copy in imports from source files in library that MATLAB won't load
- comment out suspect imports until MATLAB call into module succeeds
- rename the module and repeat (once successfully loaded, MATLAB has to restart to see new changes)
I only had to do this 29 times to isolate the following problematic imports in sima
. :)
from PIL import Image
from skimage.measure import approximate_polygon
from skimage.measure import find_contours
from skimage.filter import threshold_otsu
from skimage import transform as tf
from sklearn.decomposition import FastICA
import scipy.io
from scipy.stats import uniform
from scipy.linalg import inv
from scipy.sparse.linalg import eigsh, eigs
from scipy.sparse.linalg import inv
from scipy import ndimage
from scipy.ndimage import measurements
import cv2
# note the following are OK:
from scipy.linalg import eig, eigh, norm
from scipy import sparse
from multiprocessing import Pool
import multiprocessing.util as util
running this code in python shows no import exceptions:
lines = """
import h5py
from PIL import Image
from skimage.measure import approximate_polygon
from skimage.measure import find_contours
from skimage.filter import threshold_otsu
from skimage import transform as tf
from sklearn.decomposition import FastICA
import scipy.io
from scipy.stats import uniform
from scipy.linalg import inv
from scipy.sparse.linalg import eigsh, eigs
from scipy.sparse.linalg import inv
from scipy import ndimage
from scipy.ndimage import measurements
import cv2
"""
def tryImport(i):
try:
exec(i)
except ImportError as e:
print i
print e
print
map(tryImport, filter(lambda x: len(x) > 0, lines.splitlines()))
but in MATLAB, it gives the output:
import h5py
cannot import name _errors
from PIL import Image
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PIL/_imaging.so, 2): Library not loaded: /opt/local/lib/libtiff.5.dylib
Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PIL/_imaging.so
Reason: Incompatible library version: _imaging.so requires version 8.0.0 or later, but libtiff.5.dylib provides version 6.0.0
from skimage.measure import approximate_polygon
cannot import name _ni_support
from skimage.measure import find_contours
cannot import name _ni_support
from skimage.filter import threshold_otsu
cannot import name _ni_support
from skimage import transform as tf
cannot import name hough_ellipse
from sklearn.decomposition import FastICA
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/optimize/_lbfgsb.so, 2): Symbol not found: __gfortran_transfer_character_write
Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/optimize/_lbfgsb.so
Expected in: /Applications/MATLAB_R2015a.app/sys/os/maci64/libgfortran.3.dylib
in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/optimize/_lbfgsb.so
import scipy.io
cannot import name doccer
from scipy.stats import uniform
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so, 2): Symbol not found: __gfortran_stop_numeric_f08
Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so
Expected in: /Applications/MATLAB_R2015a.app/sys/os/maci64/libgfortran.3.dylib
in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so
from scipy.sparse.linalg import eigsh, eigs
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so, 2): Symbol not found: __gfortran_transfer_array_write
Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so
Expected in: /Applications/MATLAB_R2015a.app/sys/os/maci64/libgfortran.3.dylib
in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so
from scipy.sparse.linalg import inv
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so, 2): Symbol not found: __gfortran_transfer_array_write
Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so
Expected in: /Applications/MATLAB_R2015a.app/sys/os/maci64/libgfortran.3.dylib
in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so
from scipy import ndimage
cannot import name _ni_support
from scipy.ndimage import measurements
cannot import name _ni_support
import cv2
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cv2.so, 2): Library not loaded: /opt/local/lib/libtiff.5.dylib
Referenced from: /opt/local/lib/libopencv_highgui.2.4.dylib
Reason: Incompatible library version: libopencv_highgui.2.4.dylib requires version 8.0.0 or later, but libtiff.5.dylib provides version 6.0.0
i found that this method shows the errors from matlab:
>> py.importlib.import_module('sima')
Python Error: cannot import name hough_ellipse
so is there any hope in proceeding from here?
I don't really understand the errors, as best I can surmise, some of them are:
matlab's libtiff.5.dylib provides version 6.0.0, but PIL and opencv require >= 8.0.0
matlab's libgfortran.3.dylib doesn't work for scipy's _lbfgsb.so, _ufuncs.so, _arpack.so
would there be some way to compile scipy/PIL/opencv
against MATLAB's libraries?
would it be extremely painful? :)
i've found the following 3rd party python-from-matlab bridges:
- http://algoholic.eu/matpy/ -- numpy.ndarray integration
- https://github.com/kw/pymex -- numpy.ndarray integration, less recent development, but also looks good
- https://github.com/pv/pythoncall -- even older, looks less polished
- https://code.google.com/p/python-matlab-wormholes/ -- also old, ndarray's only, bidirectional over TCP
are there others to consider? opinions/experience on which is most promising for this scenario? i'm guessing matpy...
pymex was no good
matpy is no good:
>> py_shell
py> import sima
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/__init__.py", line 6, in <module>
from sima.imaging import ImagingDataset
File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/imaging.py", line 26, in <module>
import h5py
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/h5py/__init__.py", line 23, in <module>
from . import _conv
File "h5py/h5t.pxd", line 14, in init h5py._conv (/opt/local/var/macports/build/_opt_mports_dports_python_py-h5py/py27-h5py/work/h5py-2.5.0/h5py/_conv.c:6958)
File "h5py/h5t.pyx", line 139, in init h5py.h5t (/opt/local/var/macports/build/_opt_mports_dports_python_py-h5py/py27-h5py/work/h5py-2.5.0/h5py/h5t.c:20282)
File "h5py/h5t.pyx", line 73, in h5py.h5t.lockid (/opt/local/var/macports/build/_opt_mports_dports_python_py-h5py/py27-h5py/work/h5py-2.5.0/h5py/h5t.c:2562)
File "h5py/h5t.pyx", line 42, in h5py.h5t.typewrap (/opt/local/var/macports/build/_opt_mports_dports_python_py-h5py/py27-h5py/work/h5py-2.5.0/h5py/h5t.c:2196)
RuntimeError: Interface initialization failed (Not a datatype object)
py> import sima
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/__init__.py", line 6, in <module>
from sima.imaging import ImagingDataset
File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/imaging.py", line 33, in <module>
import sima.misc
File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/misc/__init__.py", line 21, in <module>
from skimage import transform as tf
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/skimage/transform/__init__.py", line 1, in <module>
from ._hough_transform import (hough_ellipse, hough_line,
File "skimage/transform/_hough_transform.pyx", line 13, in init skimage.transform._hough_transform (skimage/transform/_hough_transform.c:22321)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/skimage/draw/__init__.py", line 2, in <module>
from .draw3d import ellipsoid, ellipsoid_stats
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/skimage/draw/draw3d.py", line 3, in <module>
from scipy.special import (ellipkinc as ellip_F, ellipeinc as ellip_E)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/__init__.py", line 586, in <module>
from ._ufuncs import *
ImportError: dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so, 2): Symbol not found: __gfortran_stop_numeric_f08
Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so
Expected in: /Applications/MATLAB_R2015a.app/sys/os/maci64/libgfortran.3.dylib
in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/_ufuncs.so
py> import sima
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/__init__.py", line 6, in <module>
from sima.imaging import ImagingDataset
File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/imaging.py", line 33, in <module>
import sima.misc
File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/misc/__init__.py", line 21, in <module>
from skimage import transform as tf
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/skimage/transform/__init__.py", line 1, in <module>
from ._hough_transform import (hough_ellipse, hough_line,
ImportError: cannot import name hough_ellipse
py> import sima
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/__init__.py", line 6, in <module>
from sima.imaging import ImagingDataset
File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/imaging.py", line 33, in <module>
import sima.misc
File "/Users/eflister/Library/Python/2.7/lib/python/site-packages/sima/misc/__init__.py", line 21, in <module>
from skimage import transform as tf
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/skimage/transform/__init__.py", line 1, in <module>
from ._hough_transform import (hough_ellipse, hough_line,
ImportError: cannot import name hough_ellipse
py> import picos.tools
py> print picos.tools.available_solvers()
['cvxopt', 'mosek7']
i'm starting to think wormholing to a separate process sounds good...
but what about using LD_PRELOAD
to start matlab with updated libgfortran and libtiff?
the versions i want are located:
/opt/local/lib/libgcc/libgfortran.3.dylib
/opt/local/lib/libtiff.5.dylib
from within a fresh matlab session, the following didn't help.
>> setenv('DYLD_LIBRARY_PATH',['/opt/local/lib:' getenv('DYLD_LIBRARY_PATH')])
>> getenv('DYLD_LIBRARY_PATH')
ans =
/opt/local/lib:/Applications/MATLAB_R2015a.app/sys/os/maci64:/Applications/MATLAB_R2015a.app/bin/maci64/../../Contents/MacOS:/Applications/MATLAB_R2015a.app/bin/maci64:/Applications/MATLAB_R2015a.app/extern/lib/maci64:/Applications/MATLAB_R2015a.app/runtime/maci64:/Applications/MATLAB_R2015a.app/sys/java/jre/maci64/jre/lib/./native_threads:/Applications/MATLAB_R2015a.app/sys/java/jre/maci64/jre/lib/./server:/Applications/MATLAB_R2015a.app/sys/java/jre/maci64/jre/lib/./lib/jli
but!!!
~$ DYLD_INSERT_LIBRARIES=/opt/local/lib/libgcc/libgfortran.3.dylib:/opt/local/lib/libtiff.5.dylib /Applications/MATLAB_R2015a.app/bin/matlab
solves all the import problems!!!
>> import py.sima
Error using import
Import argument 'py.sima' cannot be found or cannot be imported.
>> import py.sima.Sequence % succeeds!
>> import py.sima.ImagingDataset % succeeds!
>> x = py.sima.ImagingDataset([1,2],'~')
Python Error: ImagingDataset objects must be initialized with a list of sequences.
>> x = py.numpy.random.random([2,3,4])
x =
Python ndarray with properties:
T: [1x1 py.numpy.ndarray]
base: [1x1 py.NoneType]
ctypes: [1x1 py.numpy.core._internal._ctypes]
data: [1x192 py.buffer]
dtype: [1x1 py.numpy.dtype]
flags: [1x1 py.numpy.flagsobj]
flat: [1x1 py.numpy.flatiter]
imag: [1x1 py.numpy.ndarray]
itemsize: 8
nbytes: 192
ndim: 3
real: [1x1 py.numpy.ndarray]
shape: [1x3 py.tuple]
size: 24
strides: [1x3 py.tuple]
[[[ 0.4158531 0.18912596 0.20743597 0.92631168]
[ 0.0371785 0.48093953 0.88221637 0.85687334]
[ 0.2267605 0.79464779 0.97438662 0.47791076]]
[[ 0.05602249 0.06596678 0.05691491 0.02706325]
[ 0.28477145 0.74781928 0.56700236 0.14500885]
[ 0.87999554 0.78932748 0.24473039 0.18383564]]]
>> x = py.sima.Sequence.create('ndarray',x)
The class "py.sima.Sequence" is undefined.
Perhaps Java is not running.
>> py.sima.segment.STICA()
ans =
Python STICA with no properties.
<sima.segment.stica.STICA object at 0x138e15890>
>> py.sima.segment.PlaneCA1PC()
ans =
Python PlaneCA1PC with no properties.
<sima.segment.ca1pc.PlaneCA1PC object at 0x138e158d0>
we are finally SO CLOSE. but why can't i get sima.Sequence.create()
to run!??!