0

I only get this error with anaconda3 and swig on a Mac. Does anyone have any suggestions on how to resolve this?

This is the test.i file.

# test.i
%module test

%{
int hello();
%}

This is the test.c file.

//test.c
#include <stdio.h>

int hello() {

    printf("Hello\n");
    return 0;

}

This is the compilation steps for creating the extension.

$ swig -python test.i
$ cc -c $(python3-config  --cflags) test.c test_wrap.c
$ cc -bundle -L/Users/$USER/miniconda3/lib/python3.6/config-3.6m-darwin -lpython3.6m -ldl test.o test_wrap.o -o _test.so

$ python test.py
Fatal Python error: PyThreadState_Get: no current thread

[1]    97445 abort      python test.py

Again, there's no error with any other operating system. They corresponding steps work. It works with Homebrew Python2 and works with Homebrew Python3. It also works with Anaconda2. But it does not work with Anaconda3 or a Anaconda3 environment.

See below for a minimal working example.

https://github.com/kdheepak/mwe-swig-python3-anaconda

kdheepak
  • 1,274
  • 11
  • 22

2 Answers2

2

The problem is likely that you are trying to link against the python3.6 library whereas the python executable already has the python library linked in.

Try doing the same as what a setup.py would do (this works fine with your github example):

#!/usr/bin/env python
from distutils.core import setup, Extension
test_module = Extension('_test', sources=['test_wrap.c', 'test.c'],)
setup (name = 'test', version = '0.1', ext_modules = [test_module], py_modules = ["test"],)

You will see that this creates a link command that uses -undefined dynamic_lookup without explicitly linking against the python3.6m or ld libraries.

Sander Niemeijer
  • 36
  • 2
  • Thanks for the answer. Changing the cmake file to use undefined dynamic lookup fixed this. Can you tell me how you figured out that that's what distutils was doing? – kdheepak Jan 24 '18 at 23:47
  • If you just run the setup.py script (trying to build or install the extension) then you should see the compilation and link commands appear in your terminal. – Sander Niemeijer Jan 26 '18 at 08:49
  • I see. I used `pip install -e .` and didn't see that command when I tried it. – kdheepak Jan 27 '18 at 10:09
1

python3-config --cflags should not be used when creating an extension module for Python, nor should you always link to a Python library. Instead you should query Python itself (in particular, the distutils sysconfig module) to get the appropriate values.

The compiler flags should be queried via: python -c "from distutils import sysconfig; print(sysconfig.get_config_vars('CFLAGS'))"

The linker flags should be queried via: python -c "from distutils import sysconfig; print(sysconfig.get_config_var('BLDSHARED').split(' ', 1)[1])"

Ray Donnelly
  • 3,920
  • 1
  • 19
  • 20
  • Thanks for the answer. At this point, I'm just using `-undefined dynamic_lookup` to solve this. Would you recommend your approach over this? – kdheepak Apr 04 '18 at 15:14
  • `-undefined dynamic_lookup` is not a way to solve this, no. That only prevents the (static) linker (ld64) from complaining about missing symbols. The problem here is that there are two providers (the `python` executable and the `libpython.dylib` shared library) of the same symbol (`PyThreadState_Get`) and the one from the dylib gets called instead of the one from the executable when instead it simply shouldn't have been loaded at all. – Ray Donnelly Apr 16 '18 at 00:34
  • I have already detailed how to solve this. You simply need to stop passing `-lpython3.6m` and instead ask Python itself what the linker flags should be. – Ray Donnelly Apr 17 '18 at 10:06
  • Ah thanks. I was replying from my phone and didn't realize this was part of the thread. I'll try your approach and see if that works. – kdheepak Apr 17 '18 at 19:35