Cython Link Errors for OSX

Question

I have a very similar issue to this SO post; however, after implementing the proposed fix there - I still get the below error.

I'm trying to compile a C file that's been generated by Cython.

cython ConnectFour.pyx --embed

I try to compile my c file like so:

gcc -I /usr/local/Cellar/python3/3.7.0/Frameworks/Python.framework/Versions/3.7/Headers -o ConnectFour ConnectFour.c

However I get the following errors:

Undefined symbols for architecture x86_64:
  "_PyBaseObject_Type", referenced from:
      ___Pyx_InBases in ConnectFour-3c26d7.o
  "_PyBytes_FromStringAndSize", referenced from:
      ___pyx_pymod_exec_ConnectFour in ConnectFour-3c26d7.o
      ___Pyx_InitStrings in ConnectFour-3c26d7.o
  "_PyCFunction_NewEx", referenced from:
      ___pyx_pymod_exec_ConnectFour in ConnectFour-3c26d7.o
  "_PyCFunction_Type", referenced from:
      ___Pyx_PyObject_CallOneArg in ConnectFour-3c26d7.o
      ___Pyx_PyObject_CallNoArg in ConnectFour-3c26d7.o
      ___Pyx_PyObject_Call2Args in ConnectFour-3c26d7.o
      ___Pyx_PyCFunction_FastCall in ConnectFour-3c26d7.o
      ___pyx_pf_11ConnectFour_36eval_heuristic_score in ConnectFour-3c26d7.o
      ___pyx_pf_11ConnectFour_38list_valid_col_idxs in ConnectFour-3c26d7.o
      ___pyx_pf_11ConnectFour_40deep_copy_board in ConnectFour-3c26d7.o
      ...
  "_PyCode_New", referenced from:
      ___Pyx_InitCachedConstants in ConnectFour-3c26d7.o
      ___Pyx_CreateCodeObjectForTraceback in ConnectFour-3c26d7.o
  "_PyDict_Copy", referenced from:
      ___pyx_pw_11ConnectFour_7profile_1wrapper in ConnectFour-3c26d7.o
      ___pyx_pw_11ConnectFour_5timer_1wrapper in ConnectFour-3c26d7.o
  "_PyDict_GetItemString", referenced from:
      ___pyx_pymod_exec_ConnectFour in ConnectFour-3c26d7.o
  "_PyDict_GetItemWithError", referenced from:
      ___Pyx_PyDict_GetItem in ConnectFour-3c26d7.o
  "_PyDict_New", referenced from:
      ___pyx_pymod_exec_ConnectFour in ConnectFour-3c26d7.o
      ___Pyx_Import in ConnectFour-3c26d7.o
      ___Pyx_CyFunction_get_dict in ConnectFour-3c26d7.o
      ___Pyx_CyFunction_get_annotations in ConnectFour-3c26d7.o
      ___pyx_pf_11ConnectFour_28count_players in ConnectFour-3c26d7.o
      ___pyx_pf_11ConnectFour_52num_in_a_row in ConnectFour-3c26d7.o

I appears based on this and this SO post that I'm missing some library links? So I tried the following.

gcc -I /usr/local/Cellar/python3/3.7.0/Frameworks/Python.framework/Versions/3.7/Headers -o ConnectFour ConnectFour.c -l/usr/local/Cellar/python3/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/libpython3.7.dylib

And now I get a lib not found error.

ld: library not found for -l/usr/local/Cellar/python3/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/libpython3.7.dylib

I'm totally out of my depth here reading all these SO posts (first time dealing with C) so will appreciate step by step answers.

What do I need to have in order to compile and then run my C file without error-ing out?

I know that my headers are located in /usr/local/Cellar/python3/3.7.0/Frameworks/Python.framework/Versions/3.7/Headers and the library is located in /usr/local/Cellar/python3/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/libpython3.7.dylib

I'm on OSX and I installed Python using Homebrew.

Don't know much about MacOS, but doesn't it have `python-config`-utility which can be used as described here: https://stackoverflow.com/a/45424720/5769463 — ead, Apr 05 '19 at 05:00
Yeah I read that post but it seems unclear what I need actually do - barely understand the answer that is there haha — ptk, Apr 05 '19 at 05:04
So maybe you should start with a simple module, which is can be build as described here https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html and compare the commands used by, for example, cythonize — ead, Apr 05 '19 at 05:09
Ahh yes I’ve actually done that. I managed to run my script from python but I’m trying to go the next level which is to just have an executable from C to run but figuring it out has been a nightmare... — ptk, Apr 05 '19 at 05:12
Hey @ead, your link turned out to be a great help. I gave it proper read and figured it out :) Posted my simplified solution - feel free to edit it if you wish. Thanks for the help. — ptk, Apr 08 '19 at 11:40

score 3 · Accepted Answer · answered Apr 08 '19 at 11:39

So I ended up reading this SO post more carefully per @ead's suggestions and it turned out to be a big help. Writing the following for those who encounter a similar problem.

After running cython ConnectFour.pyx --embed my goal was to compile the .c file into an executable. This means (1) need to build .o object files and (2) convert the .o object file into an executable that is runnable like so: ./MyExecutable

Step 1. Building .o object files

We need to run the following but the problem I encountered was finding out how to obtain my flags.

gcc -c hello.c -o hello.o <our cflags>

Since I installed Python via Homebrew, the Python and its header files are located in the following directory for me.

/usr/local/Cellar/python3/3.7.0/Frameworks/Python.framework/Versions/3.7/bin/

In order to generate the cflags for me - I had to run the following command. Note the --cflags flag. The command runs the config utility for my Python 3 installation, which is installed in the weird directory (since I installed Python via Homebrew).

/usr/local/Cellar/python3/3.7.0/Frameworks/Python.framework/Versions/3.7/bin/python3.7-config
--cflags

Below is a snippet of the flags it generated for me.

-I/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/include/python3.7m -I/usr/local.......etc etc.

Using the flags I ran the following to generate my object files.

gcc -c ConnectFour.c -o ConnectFour.o -I/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/include/python3.7m -I/...etc etc.

Step 2. Convert the object file into an executable

In order to generate an executable, we need to run the following.

gcc main.o hello.o -o prog <our ldflags>

To generate the ldflags, this time I needed to run the following command. Again we are using the config utility but this time with --ldflags.

/usr/local/Cellar/python3/3.7.0/Frameworks/Python.framework/Versions/3.7/bin/python3.7-config --ldflags

This generated the following flags for me.

-L/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/config-3.7m-darwin -lpython3.7m ...etc. etc.

Using those flags, I ran the following to generate my executable.

gcc ConnectFour.o -o ConnectFour -L/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/config-3.7m-darwin -lpython3.7m ...etc. etc.

Done! An executable file popped up and I ran it like any other C executable.

Special thanks to @ead for pointing me in the right direction.

score 1 · Answer 2 · answered Apr 06 '19 at 13:54

Try running

gcc -I /usr/local/Cellar/python3/3.7.0/Frameworks/Python.framework/Versions/3.7/Headers -o ConnectFour ConnectFour.c -L/usr/local/Cellar/python3/3.7.0/Frameworks/Python.framework/Versions/3.7 -lpython3.7

The -L flag adds that directory to the search path and then use -l to link (since -l doesn't seem to accept absolute paths), on Linux atleast the lib prefix and extension is omitted when using -l, i have never used a MacOSX though so try different combinations if that didn't work.

Cython Link Errors for OSX

2 Answers2