cython prints wrong values of numbers bigger than 65535

Question

I recognized a strange behavior of numbers passed to a function in cython with a value bigger than 65535. But this only appears if a pass those numbers to a function. If I define them like

cdef long long a = 145574697

everything works fine. To clarify my issue I pass the two dummy functions I used to investigate this problem

def bigNum1():
    cdef long long a = 500000
    cdef long long b = 10547746498
    cdef long long c = 65536
    cdef long long d = 65535
    print(a, b, c, d)


def bigNum2(long long a, long long b, long long c, long long d):
    print(a, b, c, d)

And the "setup.py" which was called to get the ".pyd" file.

from distutils.core import setup
from Cython.Build import cythonize


setup(
    ext_modules = cythonize("bigNumbers.pyx")
)

Afterwards I created the ".pyd" file with the command

python setup.py build_ext --inplace

using the command prompt. The used c compiler was gcc.

If I then call either "bigNum1" or "bigNum2" the following output will result.

bigNum1: (500000, 10547746498, 65536, 65535)
bigNum2: (41248, 442029762, 0, 65535).

As you see, all numbers bigger than 65535 are displayed wrong, using bigNum2. Followed you see the call of this functions.

import bigNumbers

bigNumbers.bigNum1()

a = 500000
b = 10547746498
c = 65536
d = 65535


bigNumbers.bigNum2(a, b, c, d)

I hope you understand my issue. My guess is, that I made a wrong declartion in bigNum2 that results in a wrong type or I have to make some kind of typecast before I pass the number to this method.

EDIT:

This is the text displayed during the process of the cmd prompt

python setup.py build_ext --inplace
Compiling bigNumbers.pyx because it changed.
[1/1] Cythonizing bigNumbers.pyx
running build_ext
building 'bigNumbers' extension
D:\WinPython3610\cgg64Bit\bin\gcc.exe -mdll -O -Wall -ID:\WinPython3610\python-3
.6.1.amd64\include -ID:\WinPython3610\python-3.6.1.amd64\include -c bigNumbers.c
 -o build\temp.win-amd64-3.6\Release\bignumbers.o
writing build\temp.win-amd64-3.6\Release\bigNumbers.cp36-win_amd64.def
D:\WinPython3610\cgg64Bit\bin\gcc.exe -shared -s build\temp.win-amd64-3.6\Releas
e\bignumbers.o build\temp.win-amd64-3.6\Release\bigNumbers.cp36-win_amd64.def -L
D:\WinPython3610\python-3.6.1.amd64\libs -LD:\WinPython3610\python-3.6.1.amd64\P
Cbuild\amd64 -lpython36 -lvcruntime140 -o L:\User\neon3_worksp
ace\CythonSource\src\bigNumbers.cp36-win_amd64.pyd

EDIT2:

I use the Winpython package "WinPython 3.6.1.0Qt5-64bit" which comes with cython 0.25.2. But I upgraded it with pip to gain the newest version.

First of all i tried to get my .pyd using the tutorial here.

http://docs.cython.org/en/latest/src/quickstart/build.html

But during the build I got an error message "error: Unable to find vcvarsall.bat". That why I decided to use Mingw with gcc. But this also created an error, which is described on this post.

ValueError: Unknown MS Compiler version 1900

And I solved it with the answer of Indrajit Kanjilal. And there we are now. I can create a .pyd file, can call the function, but if the value is bigger than 65535 the error occurs. Hope that helps.

Expected, but I cannot reproduce either on Linux Python3.5/Cython 0.23.4/gcc 6. — Pierre de Buyl, Oct 12 '17 at 07:26
How about defining bigNum2 as a C function with `cdef bigNum2`? — Oluwafemi Sule, Oct 12 '17 at 07:42
Marcel S., the result you obtain suggests strongly that at some point there was a mixup between the Python install, the Cython install and the development environment. Can you give the installation method for all of these components? — Pierre de Buyl, Oct 12 '17 at 07:48
First of all, thanks for your fast replies. Oluwafemi Sule I tried your advise, but this wont work. Pierre de Buyl see my question, EDIT2. I will add some details about that there and how i set it all up in the next minutes — stranger0612, Oct 12 '17 at 07:59
Mixing compilers when building native code is never going to end well.. You have (a) Visual studio built Python 3.6 using (b) Mingw gcc built cython and custom sources, all linking to VS built python 3.6.. Yeah, good luck with that. Either install and rebuild with VS 2015 for use with Python 3.6 or install a whole new environment via [conda](https://conda.io/miniconda.html) with everything compiled by the same compiler. — danny, Oct 12 '17 at 15:05
Thanks for your reply. I finaly got it to work with your tip to use VS 2015. — stranger0612, Oct 12 '17 at 18:42

score 1 · Accepted Answer · answered Oct 13 '17 at 10:39

In short, don't mix compilers when linking.

The issue here is not related to Cython but with using

Visual Studio built Python distribution
MingW GCC built Python extension

and linking them together. Due to compiler implementation differences, in particular for the above Visual Studio implements long ints differently, this will break in various fun ways.

When writing Python extensions, best to always use the same compiler as was used for the Python distribution to build the extensions.

For the official Python distributions these are Visual Studio 9 to 15 for Py 2.6 to 3.6 - see Which Microsoft Visual C++ compiler to use with a specific Python version on the Python wiki.

Solution is either to use Visual Studio 15.0 (aka VS 2014, used for Python 3.6), or to use a conda environment which builds everything including Python with GCC.

cython prints wrong values of numbers bigger than 65535

1 Answers1