Find exact C++ implementation of Cython file

Question

I am trying to find out how Cython compiles .pyx into .cpp files, and how to find the exact piece of C++ translation of my .pyx file.

My file is a simple helloworld.pyx file

print("Hello world")

I use my setup.py as follows from the official Cython guide, which is as follows:

from Cython.Build import cythonize

extensions = [
    Extension(
        "helloworld",
        sources=["helloworld.pyx"],
        language="c++"
    ),
]

setup(ext_modules = cythonize(extensions))

After running python3 setup.py build_ext --inplace

This very nicely produces both a .cpp and .so file. However, I get approximately 3000 lines of code in my .cpp file, and most of it are definitions and the like. I was rather happy when I saw that the file contained the following:

/* Implementation of 'helloworld' */
static const char __pyx_k_end[] = "end";
static const char __pyx_k_file[] = "file";
static const char __pyx_k_main[] = "__main__";
static const char __pyx_k_name[] = "__name__";
static const char __pyx_k_test[] = "__test__";
static const char __pyx_k_print[] = "print";
static const char __pyx_k_Hello_world[] = "Hello world";
static const char __pyx_k_cline_in_traceback[] = "cline_in_traceback";
static PyObject *__pyx_kp_s_Hello_world;
static PyObject *__pyx_n_s_cline_in_traceback;
static PyObject *__pyx_n_s_end;
static PyObject *__pyx_n_s_file;
static PyObject *__pyx_n_s_main;
static PyObject *__pyx_n_s_name;
static PyObject *__pyx_n_s_print;
static PyObject *__pyx_n_s_test;
/* Late includes */

However, this is not valid .cpp. Is it possible to generate valid C++ code in the process of compilation and output this piece of code somehow? I am not able to copy the entire file, as it is too long, sadly.

EDIT:

A very concrete example of what I wish to achieve is the exact things these guys are doing: Generating SIMD instructions from Cython code

They write Cython, use the command as I, and then get some clean C code which is compilable.

score 1 · Accepted Answer · answered Sep 01 '22 at 16:52

First thing to make very clear: Cython does not create short self-contained code that's independently compilable. It's designed to be compiled as part of a Cython module with all the 3000+ lines of utility code that surrounds it. This is especially true for code that interacts with Python a lot (for example your print statement).

So if you are expecting to be able to copy a short function of C++ code and run it independently the answer is "you can't".

Second:

However, this is not valid .cpp. [...]

That's just wrong. It's completely valid C++. It isn't standalone (it depends on Python.h) but it is a set of declarations of global constants.

Third (actually directly answering your question) - by default Cython embeds your input Cython code into comments into your file. That makes it very easy to find the C/C++ code that matches up with the code you have it just by searching for a particular line (remember that one line may generate code in more than one place though, particularly for function definitions).

The key line in your file is

/* "hw.py":1
 * print("Hello world")             # <<<<<<<<<<<<<<
 */
  if (__Pyx_PrintOne(0, __pyx_kp_s_Hello_world) < 0) __PYX_ERR(0, 1, __pyx_L1_error)

Note that it depends on the Cython utility-code function __Pyx_PrintOne and the global string constant __pyx_kp_s_Hello_world. I.e. not standalone.

Another useful tool is the cython -a command-line option that creates an annotated HTML file showing how your input code links up with C/C++ code.

Find exact C++ implementation of Cython file

1 Answers1