1

In my project I am wrapping some C++ code (that I have full control over) with Cython to be able to call the C++ functionality from Python.

In one of the C++ header files I want to use the "extern" keyword to define (but not initialize) a variable, which is then used in the corresponding cpp implementation file. I want to set the value for that variable from within the Cython wrapper.

In the following you can find a minimal version of my code.

Edit: As pointed out by @ead, my original code sample was not suited to reproduce the problem. Here is a minimal working example that is better suited to point out the problem.

test.h

extern int testVariable;

void function1();

void function2();

test.cpp

#include "test.h"
#include <iostream>

int testVariable;

void function1() {
    std::cout << "function1 called, testVariable = " << testVariable << "\n";
}

void function2() {
    std::cout << "function2 called, testVariable = " << testVariable << "\n";
}

wrapper1.pyx

cdef extern from "test.h":

    int testVariable

    void function1()


testVariable = 42

cpdef wrapper_function1():
    function1()

wrapper2.pyx

cdef extern from "test.h":

    void function2()


cpdef wrapper_function2():
    function2()

main.py

from wrapper1 import wrapper_function1
from wrapper2 import wrapper_function2

if __name__ == '__main__':
    wrapper_function1()
    wrapper_function2()

setup.py

from Cython.Build import cythonize
from setuptools import setup, Extension

sources = [
    '*.pyx',
    'test.cpp'
]

extensions = [
    Extension(name='*', sources=sources, language='c++')
]

setup(name='test',
      packages=['test'],
      install_requires=["Cython>=0.29.0"],
      python_requires='>=3.7',
      ext_modules=cythonize(extensions, language_level='3'))

The code is compiled via the command python3.7 setup.py build_ext --inplace.

When executing the main.py, the output is as follows:

function1 called, testVariable = 42
function2 called, testVariable = 0

When calling function1 everything works as expected, i.e. testVariable is set to 42. When calling function2, which is wrapped in a separate Cython file, testVariable seems to be uninitialized, which is surprising to me as testVariable is supposed to be a global variable.

I split the wrapper across two pyx-files, because I do not actually want to call function2 from Python, but from C++ (the code is not shown here for brevity). When calling the function from C++, the problem persists, i.e., testVariable is 0.

Thanks for your answers!

PS: My code was inspired by https://stackoverflow.com/a/52925796

Michael Rapp
  • 324
  • 3
  • 8
  • Not sure, what could be your problem but your function is a `cdef` how do you call it? If I replace `cdef` through `def` (without inline) it works as expected. If you work in a jupiter-notebook or repl you could end up calling function from older versions, so restarting everything might solve you problems. – ead Aug 03 '20 at 14:17
  • I am running this locally in PyCharm. The `cdef` function is called from another Cython function that is callable from Python itself. – Michael Rapp Aug 03 '20 at 14:50
  • Actually, it is not clear to me whether the line `testVariable = 42` is executed at all, because the only thing I am invoking explicitly is the function `test_function`. – Michael Rapp Aug 03 '20 at 14:55
  • Ok, than your example isn't [mcve]. Depending on how you build the second extension you could end up with different `testVariable`s (see e.g. https://stackoverflow.com/a/45372319/5769463). However as long as you cimport the the first extension `testVariable=42` should be called. – ead Aug 03 '20 at 15:00
  • I wonder how Cython works exactly: it's not inconceivable that your `testVariable = 42` happens before static initialization is complete on the C++ side – Asteroids With Wings Aug 03 '20 at 15:49
  • "Actually, it is not clear to me whether the line testVariable = 42 is executed at all" - that's probably right. A `.pxd` is not really executable code - it's only to tell Cython about definitions elsewhere. I'm surprised the line compiles. That's probably a bug. – DavidW Aug 03 '20 at 16:29
  • @DavidW I have missed, that this is pxd and not pyx. That is definitely a problem. – ead Aug 03 '20 at 16:37
  • It turned out my example was not really suited to reproduce the problem. I updated the code to make things clearer. – Michael Rapp Aug 03 '20 at 16:55
  • 1
    Your problem is that there are two different testVariableS (you can verify it by printing the addresses in both functions): one in every extension. It is the same problem as in the link I gave you above: you should not link the same cpp-file to different extensions – ead Aug 03 '20 at 17:14

1 Answers1

0

I would solve this problem using a class, like:

header

extern int m_nfoo;


class   __declspec(dllexport)  MyCppClass
{
public:
    MyCppClass();
    ~MyCppClass(void);

    int GetFoo();
    void SetFoo(int Foo);
....

c++:

int m_nFoo=0;


int MyCppClass::GetFoo()
{ 
    return m_nFoo;
}

void MyCppClass::SetFoo(int Foo)
{ 
    m_nFoo=Foo;
}

pxd:

cdef extern from "MyCppClass.h":

     
    cdef cppclass MyCppClass:
        MyCppClass()
        int GetFoo()
        void SetFoo(int Foo)

pyx:

cdef class myClass:
    """
    wrapper of MyCppClass
    """
       
    def __cinit__(self):
        self.thisptr = new MyCppClass()
        
    def setFoo(self,foo):
        self.thisptr.SetFoo(foo)
        
    def getFoo(self):
        return self.thisptr.GetFoo()

and in some py:

    I=myClass()

    I.setFoo(42)
    print ('foo=',I.getFoo())

and that works fine

YLS
  • 73
  • 7
  • I agree that this is a viable alternative, but in my case it would be cleaner to use an extern declaration, if possible (my code is just a toy example to illustrate the issue). – Michael Rapp Aug 03 '20 at 15:07