2

Let's say I'm trying to wrap vector.

module/__init__.pxd:

from libcpp.vector cimport vector
from libc.stdint cimport uint32_t

cdef class MyVector:
    cdef vector[uint32_t]* thisptr

module/__init__.pyx:

from libc.stdint cimport uint32_t
from libcpp.vector cimport vector
from cython.operator cimport dereference as deref

cdef class MyVector:
    # the field declaration is in *.pxd
    def __cinit__(self):
        self.thisptr = new vector[uint32_t]()
        self.thisptr.push_back(42)

    def __dealloc__(self):
        del self.thisptr
        self.thisptr = <vector[uint32_t]*> NULL

    def mysize(self):
        return self.thisptr.size()

    def myget(self):
        return deref(self.thisptr)[0]

module/__init__.pyxbld and run.pyxbld:

def make_ext(modname, pyxfilename):
    from distutils.extension import Extension
    return Extension(
        name=modname,
        sources=[pyxfilename],
        language='c++'
    )

run.pyx:

from module cimport MyVector

cdef main():
    obj = MyVector()
    print(obj.thisptr.size()) # 1
    print(obj.mysize()) # 1
    print(obj.myget())  # 42

main()

test.py:

import pyximport
pyximport.install()

import run

When I run test.py, it crashes with the following traceback:

Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/pyximport/pyximport.py", line 210, in load_module
    mod = imp.load_dynamic(name, so_path)
  File "module/__init__.pxd", line 5, in init run (/home/pastafarianist/.pyxbld/temp.linux-x86_64-3.4/pyrex/run.cpp:918)
    cdef class MyVector:
AttributeError: 'module' object has no attribute 'MyVector'

The same code works if I move module/__init__.pyx to module.pyx and module/__init__.pxd to module.pxd. What am I missing and how do I fix it?

A few other related questions.

  1. Is there any way to expose a templated wrapper to Cython code, so that I can have MyVector[uint16_t] without needing to write another wrapper?
  2. Am I correct in adding a pyxbld file for each source file which interacts with C++ code? Is this redundant? (I like the convenience of pyximport and I don't want to recompile the code every time manually while I'm still trying to get it to work.)
  3. How do I compile module to a standalone extension? What should setup.py look like?
  4. In the above code I never used deref before calling C++ methods. How does Cython understand where I mean -> and where I mean .?

I would appreciate help with any of these questions.

UPD: actually, I was wrapping Google's sparsehash, and I have figured out a way to do what I wanted, but it looks like black magic. I would still appreciate a clarification of what was going on with this error and how to write Cython wrappers properly.

Pastafarianist
  • 833
  • 11
  • 27

2 Answers2

2

Your main issue is that the code in __init__.so (generated from __init__.pyx) does not get executed unless you also have __init__.py (which does not get executed in this case). Therefore, you can solve the problem by making an empty __init__.py file. This all feels like a messy edge-case. In your original code, the MyVector class would never be added to the module and so it fails to find it with the error you describe.

Short answers to your other questions are:

  1. no - unfortunately there is no good way to build a templated wrapper class. You don't need to build wrapper classes for anything you only use in Cython though.

  2. and

  3. I think these would end up being pretty significant answers in their own right and so I'm skipping them. setup.py seems to be preferred to pyximport in general, and good examples of how to use it are in the documentation.

  4. Cython keeps track of the types of objects internally. It's fairly straightforward for it to substitute -> whenever it knows it's using a pointer and . otherwise, so as you say you don't have to do so yourself.

Community
  • 1
  • 1
DavidW
  • 29,336
  • 6
  • 55
  • 86
  • Thank you. I'd like to note that I wanted a wrapper class for a more pythonic interface, even inside a Cython module. Could you provide an example of a simple Cython wrapper? I've modeled the solution that I referenced in the UPD after GSL (that's also how I figured out about `__init__.py`), but it's too complex, and I've probably overengineered something. – Pastafarianist Dec 02 '15 at 21:17
  • 1
    It looks OK - you've fitted it to work broadly like a `dict` which I think is as Pythonic as you can get. Unfortunately, Cython does involve wrapping every function independently, which gives you quite long interfaces, so I'd say next time just cut it down to the bits you need. I can't see a whole lot else. – DavidW Dec 03 '15 at 07:29
-1

You should consider using 'swig'. This tool will create all the code you need for interacting between C++ and Python. Link to swig website.

I used swig in a complex project and it worked very nicely.

Anyway, you can use the swig created code as a reference if you don't want to use this framework in your code.

egur
  • 7,830
  • 2
  • 27
  • 47