9

My question is similar to How to import a module given the full path? however I'm not importing a .py source file, I'm importing a package with a .pyd.

During runtime I am creating new package modules from some dynamically generated c code. I successfully generate package Foo with a __init__.py file and a mod.pyd:

/a/temp/dir/foo/
    __init__.py
    bar.pyd

The example code I'm working with is

import importlib.util
spec = importlib.util.spec_from_file_location("foo.bar", "/path/to/file.py")
foo = importlib.util.module_from_spec(spec)
spec.loader.exec_module(foo)
foo.bar()

If I attempt to use spec_from_file_location('foo.bar', '/a/temp/dir/foo/__init__.py') the module bar from the pyd fails to load.

If I attempt to use spec_from_file_location('foo.bar', '/a/temp/dir/foo/') then spec_from_file_location returns None.

If I attempt to use spec_from_file_location('foo.bar', '/a/temp/dir/foo/bar.pyd') I get the following error stack:

File "<frozen importlib._bootstrap>", line 577, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 903, in create_module
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

An alternative to the recommended 4 line importlib solution is to create concrete classes for MetaPathFinder and SourceLoader. My following MetaPathFinder works fine, however, I am unsure how to implement the Loader correctly. get_data return value expects some Unicode representation of python code. How do I return the package structure or the pyd contents in the loader?

class MyLoader(importlib.abc.SourceLoader):
    def __init__(self, fullname, path, pyd):
        self.path = path
        self.fullname = fullname
        self.pyd = pyd

    def get_data(self, path):
        print('*** get_data', path)
        with open(os.path.join(self.path, self.pyd), 'r') as f:
            return f.read()

    def get_filename(self, fullname):
        print('*** get_filename', fullname)
        return os.path.join(self.path, '__init__.py')

    def is_package(self):
        return True

class MyFinder(MetaPathFinder):
    def __init__(self, this_module, workdir, pyd):
        self._this_module = this_module
        self._workdir = workdir
        self._pyd = pyd

    def find_spec(self, fullname, path, target=None):
        print('find_spec', fullname, path, target, self._this_module)

        if fullname != self._this_module:
            return

        filename = os.path.join(self._workdir, self._this_module)

        spec = ModuleSpec(
            name = fullname,
            loader = MyLoader(fullname, filename, self._pyd),
            origin = filename,
            loader_state = 1234,
            is_package = True,
        )
        return spec

    def is_package(self):
        return True
Community
  • 1
  • 1
totowtwo
  • 2,101
  • 1
  • 14
  • 21

0 Answers0