1

i'm using winpython (2.7) on windows 7/64, matlab 2015a, with matlab's new python bridge. 

>> py.ctypes.util.find_library('c')

ans = 

  Python str with no properties.

    msvcr90.dll

>> py.ctypes.util.find_msvcrt()

ans = 

  Python str with no properties.

    msvcr90.dll

>> py.ctypes.CDLL(py.ctypes.util.find_library('c'))
Python Error: [Error 1114] A dynamic link library (DLL) initialization routine failed

>> x=CDLL('C:\Users\nlab\Downloads\WinPython-64bit-2.7.9.5\python-2.7.9.amd64\msvcr90.dll')
Python Error: [Error 1114] A dynamic link library (DLL) initialization routine failed

a popup also comes up: 

Microsoft Visual C++ Runtime Library
R6034 "an application has made an attempt to load the c runtime library incorrectly"

a couple other SO answers suggest it's matlab putting its incompatible copy of msvc*.dll somewhere on the path, so i removed everything not from WinPython in sys.path (just matlab's \bin\win64\ and the \Python27\site-packages\ from another python install i have):

>> py.pprint.PrettyPrinter().pprint(py.sys.path)
['',
 'C:\\Users\\nlab\\Downloads\\WinPython-64bit-2.7.9.5\\python-2.7.9.amd64\\python27.zip',
 'C:\\Users\\nlab\\Downloads\\WinPython-64bit-2.7.9.5\\python-2.7.9.amd64\\DLLs',
 'C:\\Users\\nlab\\Downloads\\WinPython-64bit-2.7.9.5\\python-2.7.9.amd64\\lib',
 'C:\\Users\\nlab\\Downloads\\WinPython-64bit-2.7.9.5\\python-2.7.9.amd64\\lib\\plat-win',
 'C:\\Users\\nlab\\Downloads\\WinPython-64bit-2.7.9.5\\python-2.7.9.amd64\\lib\\lib-tk',
 'C:\\Users\\nlab\\Downloads\\WinPython-64bit-2.7.9.5\\python-2.7.9.amd64',
 'C:\\Users\\nlab\\Downloads\\WinPython-64bit-2.7.9.5\\python-2.7.9.amd64\\lib\\site-packages',
 'C:\\Users\\nlab\\Downloads\\WinPython-64bit-2.7.9.5\\python-2.7.9.amd64\\lib\\site-packages\\FontTools',
 'C:\\Users\\nlab\\Downloads\\WinPython-64bit-2.7.9.5\\python-2.7.9.amd64\\lib\\site-packages\\win32',
 'C:\\Users\\nlab\\Downloads\\WinPython-64bit-2.7.9.5\\python-2.7.9.amd64\\lib\\site-packages\\win32\\lib',
 'C:\\Users\\nlab\\Downloads\\WinPython-64bit-2.7.9.5\\python-2.7.9.amd64\\lib\\site-packages\\Pythonwin']

there are still tons of msvc*.dll sprinkled everywhere on the system, and surely some on the PATH:

>> x = py.os.environ

x = 

  Python _Environ with properties:

    data: [1x1 py.dict]

    {'TMP': 'C:\\Users\\nlab\\AppData\\Local\\Temp', <<snip>>, 'USERPROFILE': 'C:\\Users\\nlab'}

>> cellfun(@(s)fprintf('%s\n',s),strsplit(char(x{'PATH'}),';'))
C:\Program Files\Haskell\bin
C:\Program Files\Haskell Platform\2014.2.0.0\lib\extralibs\bin
C:\Program Files\Haskell Platform\2014.2.0.0\bin
C:\Users\nlab\Downloads\WinPython-64bit-2.7.9.5\python-2.7.9.amd64
C:\Users\nlab\Downloads\opencv\build\x64\vc12\bin
C:\ProgramData\Oracle\Java\javapath
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.0\bin\
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.0\libnvvp\
C:\Program Files\ImageMagick-6.8.3-Q16
C:\Program Files (x86)\OSSBuild\GStreamer\v0.10.7\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v4.1\\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v4.1\libnvvp\
C:\Program Files (x86)\PHP\
C:\Windows\system32
C:\Windows
C:\Windows\System32\Wbem
C:\Program Files\Intel\DMIX
C:\Program Files\TortoiseSVN\bin
C:\Program Files\SlikSvn\bin\
C:\Program Files\MySQL\MySQL Server 5.5\bin
C:\Program Files (x86)\Common Files\Acronis\SnapAPI\
C:\Program Files (x86)\PostgreSQL\9.2\bin
C:\Python27
C:\Python27\Scripts
C:\Program Files\Microsoft SQL Server\110\Tools\Binn\
C:\Windows\System32\WindowsPowerShell\v1.0\
C:\Program Files (x86)\LilyPond\usr\bin
C:\Program Files (x86)\Git\cmd
C:\Program Files\Microsoft Windows Performance Toolkit\
C:\Program Files\TortoiseGit\bin
C:\Program Files (x86)\QuickTime\QTSystem\
C:\Program Files (x86)\Skype\Phone\
C:\Program Files\Mosek\7\tools\platform\win64x86\bin
C:\Program Files\Haskell Platform\2014.2.0.0\mingw\bin
C:\Users\nlab\AppData\Roaming\cabal\bin
C:\Program Files (x86)\SSH Communications Security\SSH Secure Shell
C:\Gtk+\bin

i notice that in C:\Program Files\MATLAB\R2015a\bin\win64\ we only have msvc[r|p][100|110].dll -- does that mean it won't work with a distribution of python based on msvcr90 like winpython 2.7.9.5?

Community
  • 1
  • 1
user1441998
  • 459
  • 4
  • 14
  • 3
    The initialization of msvcr90.dll calls `FindActCtxSectionString` to look for "msvcr90.dll" in the current activation context. If this call fails, then it unloads, returns `FALSE`, and the loader reports that initialization failed. Normally for Python 2.7 this activation context is provided by the manifest that's embedded in python.exe. However, you can also modify the activation context at runtime. See my answer for [How do I load a C DLL from the SXS in Python?](http://stackoverflow.com/a/27392347/205580). – Eryk Sun Jun 11 '15 at 06:11
  • thanks @eryksun, that looks promising. but i'm not familiar with any of this stuff, can you give me some background? like am i trying to point python to matlab's msvcr, or v/v, and why can't they each be happy with their own? will it be dangerous that one of them is using an msvcr that it doesn't expect? what should my manifest look like? your .py code shouldn't need changes, right? i'm really trying to get shapely working from matlab, their installation is complex so i can't really edit their cdll call -- https://github.com/Toblerity/Shapely/blob/master/shapely/libgeos.py – user1441998 Jun 11 '15 at 07:14
  • apparently i have to use matlab's -- replacing shapely's `CDLL(find_library('c'))` with `CDLL('C:\\Program Files\\MATLAB\\R2015a\\bin\\win64\\msvcr100.dll')` makes it all work -- so @eryksun, if i understand, your method would be a way to determine the correct path to use automatically? – user1441998 Jun 11 '15 at 08:30
  • huh, one of the shapely devs suggested just `ctypes.cdll.msvcrt` -- which works -- i thought that was supposed to be equivalent to `find_library('c')`? anyway, then you don't need any fancy manifest/activation context stuff... – user1441998 Jun 11 '15 at 09:55
  • It depends on what it's using the CRT to do. For example, it's wrong to use a different CRT to open files (i.e. `FILE *` and the underlying file descriptors) that need to be shared with the interpreter and other extension modules. But if it's working without crashing, then it's probably ok. – Eryk Sun Jun 11 '15 at 15:05
  • FYI, python27.dll has a manifest embedded as resource ID 2, which is how it statically loads msvcr90.dll. Since this DLL is already loaded, you can use its manifest instead of a manifest file. Set `ctx.hModule = cdll.python27._handle;` `ctx.lpResourceName = c_wchar_p(2);` `dwFlags = ACTCTX_FLAG_HMODULE_VALID | ACTCTX_FLAG_RESOURCE_NAME_VALID`. – Eryk Sun Jun 11 '15 at 15:06
  • BTW, _ctypes.pyd also depends on msvcr90.dll, so you may be wondering how you can import the `_ctypes` extension module without an activation context. You can't. The `DllMain` of python27.dll stores an activation context that gets activated by `_PyImport_GetDynLoadFunc` before it calls `LoadLibraryEx`. Unfortunately this is all private within the DLL, i.e. `_Py_ActivateActCtx` and `_Py_DeactivateActCtx` aren't exported by the DLL, else it would be trivial to call these functions using `ctypes.pythonapi`. – Eryk Sun Jun 13 '15 at 02:10

2 Answers2

0

define this as file msvc.py:

import ctypes
print ctypes.cdll.msvcrt

then in matlab:

>> py.importlib.import_module('msvc')
<CDLL 'msvcrt', handle fe190000 at 771c2b38>

ans = 

  Python module with properties:

    ctypes: [1x1 py.module]

    <module 'msvc' from 'msvc.pyc'>

>> x = py.ctypes.cdll

x = 

  Python LibraryLoader with properties:

    msvcrt: [1x1 py.ctypes.CDLL]

    <ctypes.LibraryLoader object at 0x00000000771D6278>

>> x.msvcrt

ans = 

  Python CDLL with no properties.

    <CDLL 'msvcrt', handle fe190000 at 771c2b38>

i don't know why it's necessary to use msvc.py -- just importing ctypes does not give you a cdll with an msvcrt property.

user1441998
  • 459
  • 4
  • 14
  • Your MATLAB interface can't handle the dynamic `__getattr__` design of `cdll`. Can't you simply use `msvcrt = py.ctypes.CDLL('msvcrt')` to load the DLL? `cdll` caches the result on itself using built-in `setattr`. – Eryk Sun Jun 11 '15 at 15:14
0

@eryksun's method works.

the question remains, why are ctypes.cdll.msvcrt and find_library('c') implemented the way they are? they should just use @eryksun's method to return the dll in python's manifest, right?

from ctypes import *
from ctypes.wintypes import *

kernel32 = WinDLL("kernel32")

ACTCTX_FLAG_PROCESSOR_ARCHITECTURE_VALID = 0x001
ACTCTX_FLAG_LANGID_VALID = 0x002
ACTCTX_FLAG_ASSEMBLY_DIRECTORY_VALID = 0x004
ACTCTX_FLAG_RESOURCE_NAME_VALID = 0x008
ACTCTX_FLAG_SET_PROCESS_DEFAULT = 0x010
ACTCTX_FLAG_APPLICATION_NAME_VALID = 0x020
ACTCTX_FLAG_HMODULE_VALID = 0x080
DEACTIVATE_ACTCTX_FLAG_FORCE_EARLY_DEACTIVATION = 1

INVALID_HANDLE_VALUE = HANDLE(-1).value
ULONG_PTR = WPARAM  # pointer-sized unsigned integer

class ACTCTX(Structure):
    _fields_ = (("cbSize", ULONG),
                ("dwFlags", DWORD),
                ("lpSource", LPCWSTR),
                ("wProcessorArchitecture", USHORT),
                ("wLangId", LANGID),
                ("lpAssemblyDirectory", LPCWSTR),
                ("lpResourceName", LPCWSTR),
                ("lpApplicationName", LPCWSTR),
                ("hModule", HMODULE))

    def __init__(self, *args, **kwds):
        super(ACTCTX, self).__init__(sizeof(self), *args, **kwds)

CreateActCtxW = kernel32.CreateActCtxW
CreateActCtxW.restype = HANDLE
CreateActCtxW.argtypes = (POINTER(ACTCTX),)
ReleaseActCtx = kernel32.ReleaseActCtx
ReleaseActCtx.restype = None
ReleaseActCtx.argtypes = (HANDLE,)
ActivateActCtx = kernel32.ActivateActCtx
ActivateActCtx.argtypes = (HANDLE, POINTER(ULONG_PTR))
DeactivateActCtx = kernel32.DeactivateActCtx
DeactivateActCtx.argtypes = (DWORD, ULONG_PTR)

def getMsvcr90():
    ctx = ACTCTX(hModule = cdll.python27._handle 
                ,lpResourceName = c_wchar_p(2)
                ,dwFlags = ACTCTX_FLAG_HMODULE_VALID | ACTCTX_FLAG_RESOURCE_NAME_VALID
                )
    hActCtx = CreateActCtxW(byref(ctx))
    if hActCtx == INVALID_HANDLE_VALUE:
        raise WinError()

    cookie = ULONG_PTR()
    if not ActivateActCtx(hActCtx, byref(cookie)):
        raise WinError()
    msvcr90 = CDLL("msvcr90")
    if not DeactivateActCtx(0, cookie):
        raise WinError()

    ReleaseActCtx(hActCtx)

    # show DLL path
    hModule = HANDLE(msvcr90._handle)
    path = (c_wchar * 260)()    
    kernel32.GetModuleFileNameW(hModule, path, len(path))
    print(path.value)

    return msvcr90
user1441998
  • 459
  • 4
  • 14
  • It occured to me that even though `LoadLibraryW(u'msvcr90')` and `GetModuleHandleW(u'msvcr90')` won't work without the fully qualified path or an activation context, if you know for sure that your process only has one instance/version of msvcr90.dll loaded, you could instead call `EnumProcessModules` to get an array of module handles, and then call `GetModuleBaseNameW` to find the module with a base name equal to `u'msvcr90.dll'`. Then you can just wrap that in a `CDLL` instance, e.g. `msvcr90 = CDLL('msvcr90', handle=hModule)`. – Eryk Sun Jun 13 '15 at 01:41
  • Since the `_ctypes` extension module directly exposes the address of a few CRT functions, a more reliable way to get a handle to the CRT is to call `GetModuleHandleExW`. For example: `import ctypes;` `hMsvcr = ctypes.c_void_p();` `GMHEXF_FROM_ADDRESS = 4;` `ctypes.windll.kernel32.GetModuleHandleExW(GMHEXF_FROM_ADDRESS, ctypes.memset, ctypes.byref(hMsvcr));` `msvcr = ctypes.CDLL('msvcr', handle=hMsvcr.value)` – Eryk Sun Jun 13 '15 at 02:21
  • If you're having problems with the dynamic `__getattr__` usage there, try referencing the function as `ctypes.windll['kernel32']['GetModuleHandleExW']`. – Eryk Sun Jun 13 '15 at 02:26