Method not visible from C extension module

Question

I'm trying to create a simple C extension to Python and get it to run on OS X. I'm running Python 3.6. My method is apparently not getting exported from the module. Stripped down, my code looks like this:

Example.c:

#define PY_SSIZE_T_CLEAN
#include <Python.h>

static PyObject*
myfunc(PyObject* self, PyObject* args)
{
    /* Not parsing args because I'm just returning a constant */
    return PyLong_FromLong(1);
}

/* Method table */
static PyMethodDef MyMethods[] = {
    {"myfunc", myfunc, METH_VARARGS, "Just do something"},
    {NULL, NULL, 0, NULL}        /* Sentinel */
};

/* Module defintion */
static struct PyModuleDef mymodule = {
    PyModuleDef_HEAD_INIT,
    "example",   /* name of module */
    NULL, /* module documentation, may be NULL */
    -1,       /* size of per-interpreter state of the module,
                 or -1 if the module keeps state in global variables. */
    MyMethods
};

/* Module initialization */
PyMODINIT_FUNC
PyInit_example(void)
{
    //import_array(); // initialize numpy arrays
    return PyModule_Create(&mymodule);
}

I compile with

gcc -c -fPIC -I$PYINCLUDE example.c
gcc example.o -shared -o libexample.so -L$PYLIB -lpython3.6m

creating the .so file. Finally, I am trying to run this with:

import ctypes
m = ctypes.CDLL("libexample.so")
print(m.myfunc("asdf"))

when I get:

Traceback (most recent call last):
  File "test.py", line 3, in <module>
    print(m.myfunc("asdf"))
  File "/PATH/TO/anaconda3/lib/python3.6/ctypes/__init__.py", line 361, in __getattr__
    func = self.__getitem__(name)
  File "/PATH/TO/anaconda3/lib/python3.6/ctypes/__init__.py", line 366, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(0x7f96f652e4e0, myfunc): symbol not found

I realize that many tutorials recommend using setuptools or distutils to package extensions, but I'm trying to learn only one new tool at a time. Is there something obvious that I'm doing wrong?

Answer 1

The ctypes module is meant for accessing dynamically loaded libraries in regular C. You are creating a python extension. Since python functions in C are declared as static, myfunc is not exposed in your .so file. You need to compile the extension in distutils or setuptools then import the module in your python code. If you would like to use ctypes like your example, you need to rewrite your code in C similar to below.

Header file:

#ifndef _EXAMPLE_H_
#define _EXAMPLE_H_
int myfunc();
#endif

C source file:

#include "example.h"

int myfunc()
{
    /* Not parsing args because function is just returning a constant */
    return 1;
}

Answer 2

You created (or want to create and use) an extension module ([Python.Docs]: Extending Python with C or C++), which even if written in C, can / should be accessed as any other Python module.

[Python.Docs]: ctypes - A foreign function library for Python serves for as totally different purpose: dynamically loading regular (not Python modules (it can load those too, but that's a different story)) .sos (.dlls), and calling functions from them, and thus doesn't belong here. Any tweak to get it in, would be just a (lame) workaround.
There are plenty examples of how to create simple .dlls and use them from CTypes ([SO]: C function called from Python via ctypes returns incorrect value).

All in all, you should be able to call your function in 2 simple lines of code:

import example
example.myfunc()

A more elaborate example.

test_example.py:

#!/usr/bin/env python

import sys
import example


def main(*argv):
    print("Module: {:}".format(example))
    print("Functions: {:}".format(dir(example)))
    print("myfunc returned: {:}".format(example.myfunc()))


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.")
    sys.exit(rc)

Output:

[cfati@cfati-5510-0:/mnt/e/Work/Dev/StackOverflow/q069200033]> ~/sopr.sh
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***

[064bit prompt]> ls
examplemodule.c  test_example.py
[064bit prompt]> 
[064bit prompt]> gcc -fPIC -I/usr/include/python3.6 -shared -o example.cpython-36m-x86_64-linux-gnu.so -L/usr/lib/python3.6 -lpython3.6m examplemodule.c
[064bit prompt]> ls
example.cpython-36m-x86_64-linux-gnu.so  examplemodule.c  test_example.py
[064bit prompt]> 
[064bit prompt]> python3.6 test_example.py 
Python 3.6.15 (default, Sep 10 2021, 00:26:58) [GCC 9.3.0] 064bit on linux

Module: <module 'example' from '/mnt/e/Work/Dev/StackOverflow/q069200033/example.cpython-36m-x86_64-linux-gnu.so'>
Functions: ['__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'myfunc']
myfunc returned: 1

Done.

Notes:

• The name of your module should be example.so. I added that funky suffix, so that if I want to run it from multiple Python versions, I don't have to rebuild it every time I change the version, and also to be [Python]: PEP 3149 - ABI version tagged .so files compliant
  • While on this topic, you should move to newer Python versions
• I changed the name of your file to examplemodule.c (it's just a convention)
• Needless to say that it's treated as every module (including module search paths), here it wasn't the case because it's located in cwd, but you might need to add its path to ${PYTHONPATH}