Porting cython files from python2 to python3 with 2to3

Question

I have a python package which was developed under python2.7, but I need to port it to python3.6 . I use cython in some parts of the code, hence the package has both .py and .pyx files.

I tried the 2to3 command, but I got an error that I couldn't neither understand nor solve.

Example: I have the following test.pyx file

# cython: profile=False
cimport cython

@cython.boundscheck(False)
@cython.wraparound(False)
@cython.profile(False)
cpdef sillyfunction():
    print 'Thank you for your kind help'
    return

and I run 2to3 test.pyx. What I obtain is:

user@machine:~$ 2to3 test.pyx
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Can't parse test.pyx: ParseError: bad input: type=1, value=u'cython', context=(u' ', (2, 8))
RefactoringTool: No files need to be modified.
RefactoringTool: There was 1 error:
RefactoringTool: Can't parse test.pyx: ParseError: bad input: type=1, value=u'cython', context=(u' ', (2, 8))

Answer 1

You shouldn't need to do anything. Cython accepts an argument language_level (see http://cython.readthedocs.io/en/latest/src/reference/compilation.html#compiler-directives) which controls where it interprets the code as Python 2 or Python 3 (for example print as a function or as a statement).

Whichever you do the code it generates should be compilable to use with Python 2 or Python 3 (this is determined by what headers you include, which is largely arranged by the build process). There are a lot of preprocessor #if PY_MAJOR_VERSION >= 3 sections in the generated C code to ensure this.

I suspect that there are some limitations on this compatibility, and I certainly wouldn't expect all Python 3 features to work perfectly when compiled against Python 2, but as a general rule you should be able to take your existing Cython code, run Cython on it with language_level=2 (the default) and then compile it using the Python 3 headers/libraries (which setup.py should take care of by default) and it should work. There may be small, specific issues you have to work round though.