I am working on wrapping a Fortran module in Python. I chose to do it with use of Cython. My problem is passing a np.ndarray to Fortran. I am able to receive an np.ndarray from Fortran, but all my attempts to passing to Fortran didn't work.
I figured out, that the problem lays directly on the Cython - Fortran interface, as my Fotran subroutine is working properly (as much as it can work without data). The Cython side seems to be working properly too, I can manipulate variables over there.
My minimum working example:
PATTERN_wrap.f90
module PATTERN_wrap
use iso_c_binding, only: c_float, c_double, c_short, c_int
implicit none
CONTAINS
subroutine c_pattern(scalar_variable, array_variable, return_array) bind(c)
implicit NONE
INTEGER(c_int), intent(in) :: scalar_variable
INTEGER(c_int), intent(in), DIMENSION(10, 15) :: array_variable
REAL(c_float), INTENT(OUT), DIMENSION(10) :: return_array
write(*,*) "start fortran"
write(*,*) "scalar_variable"
write(*,*) scalar_variable
write(*,*) "array_variable"
write(*,*) array_variable
return_array = 3
write(*,*) "end fortran"
! call DO_PATTERN(&
! scalar_variable=scalar_variable, &
! array_variable=array_variable, &
! return_array=return_array)
!
end subroutine
end module PATTERN_wrap
Note: The call to subroutine DO_PATTERN that actually does something is commented out because it's not relevant at this moment. I just wanted to point out that code above is a wrapper.
pattern.pyx
#cython: language_level=3
import cython
import numpy as np
cimport numpy as np
cdef extern:
void c_pattern(
int *scalar_variable,
int *array_variable,
float *return_array
)
def run_pattern(
int scalar_variable,
):
cdef:
np.ndarray[int, ndim=2, mode="fortran"] array_variable = np.ones((10,15), dtype=np.int32, order='F')
np.ndarray[float, ndim=1, mode="fortran"] return_array = np.zeros(10, dtype=np.float32, order='F')
c_pattern(
&scalar_variable,
&array_variable[0,0],
&return_array[0],
)
print('Cython side')
print(return_array)
return return_array
setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy
npy_include_dir = numpy.get_include()
ext_modules = [Extension("pattern", ["pattern.pyx"],
include_dirs = [npy_include_dir],
libraries = ['gfortran', 'fftw3'], # need to include gfortran as a library
extra_link_args=[
"PATTERN_wrap.o"
])]
setup(name = 'pattern',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules)
I am compiling my fortran code with
gfortran -Wall -fbounds-check -lm -g -fbacktrace -fcheck=all -Wall -ffpe-trap=zero,invalid,overflow -fPIC -L/usr/lib/ -lfftw3 -L/usr/lib/ -lfftw3 -c PATTERN_wrap.f90
and compiling the Cython code with either python -m pip install . or python setup.py build_ext --inplace. That does not seem to have any difference.
I test the package:
$ python -c "import pattern; pattern.run_pattern(2);"
start fortran
scalar_variable
2
array_variable
end fortran
Cython side
[3. 3. 3. 3. 3. 3. 3. 3. 3. 3.]
As you can see, scalar is being passed to fortran properly, returning array is also passed back to Cython properly. The only thing not working is passing arrays from Cython to Fortran. In short, there should be an 2D array of ones printed after array_variable.
Apart of the MWE above, I tried different approaches:
passing the array with
<int*> array_variable.dataThis is discouraged by Cython (https://github.com/cython/cython/wiki/tutorials-NumpyPointerToC)Creating variable as Fortran contiguous MemoryView
int[::1,:] array_variable = np.ones((10,15), dtype=np.int32, order='F').
All my attempts failed in the same way as MWE.
I tried also using a header file, doesn't make a difference. Header file was used for example here: Fortran - Cython Workflow This question itself does not contain answer for my question - only scalars are passed to Fortran there.
I would also like to note that the same wrapper plus all underlying files are working properly when I compile a package with f2py. The subroutine also works inside of the original Fortran program.
EDIT:
My development environment is running in docker. The baseimage is continuumio/miniconda3:4.8.2 which on the other hand is based on Debian Buster.
I tested there gfortran-8 and gfortran-9 as well as a hdf5 compiler with enabled fortran. The result was all the time the same.
I decided to run my tests on my host system, Ubuntu 18.04 with gcc/gfortran 7.50. It did work properly. So I went to try different gcc versions.
I tested images:
- gcc:7
- gcc:8
- gcc:9
- gcc:10
running them with:
docker run --rm -v ~/minimum_working_example:/mwe -it gcc:7 /bin/bash
and then
apt update && apt install python3-pip -yy && cd /mwe && python3 -m pip install cython numpy && make && python3 setup.py build_ext --inplace && python3 -c "import pattern; pattern.run_pattern(2);" && rm -rf build/ *.so *.c *.mod *.o
On all of those images my code is working properly.
EDIT2:
I just ran test on bare continuumio/miniconda3:4.8.2, with the same test command (with added apt install gfortran since there is no fortran by default) and the code works.
I rebuilt my image and tested in the same way. It doesn't work...
