Issues running f2py with main .f90 module calling specific .f file? (unknown_subroutine?)

Question

I am trying to use f2py to integrate fortran functions with my main python code. However, when I try to include a specific '.f' file, f2py raises an error (but works fine with other '.f' files). I have created the following minimum working example of my main '.f90' file:

module min_example
  implicit none

  public :: calc_min

contains

  subroutine calc_min
    print*, 'test'
  return
  end subroutine calc_min

end module min_example

And the '.f' file that is causing me problems is 'qromb.f' as seen here: https://github.com/david-deboer/cosmo/blob/master/Komatsu/mf_jenkins/qromb.f

The error looks like this (sorry for the huge block - I'm new to this and not sure what is relevant):

running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "min_example" sources
f2py options: []
f2py:> /tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c
creating /tmp/tmpQULhjN/src.linux-x86_64-2.7
Reading fortran codes...
    Reading file 'qromb.f' (format:fix,strict)
Line #1 in qromb.f:"      SUBROUTINE qromb(func,a,b,ss,q) ! one parameter"
    analyzeline: No name/args pattern found for line.
Line #22 in qromb.f:"      SUBROUTINE trapzd(func,a,b,s,n,q) ! one paramete"
    analyzeline: No name/args pattern found for line.
    Reading file 'min_example.f90' (format:free)
Post-processing...
    Block: min_example
            Block: unknown_subroutine
            Block: unknown_subroutine
            Block: polint
            Block: min_example
                Block: calc_min
Post-processing (stage 2)...
    Block: min_example
        Block: unknown_interface
            Block: unknown_subroutine
            Block: unknown_subroutine
            Block: polint
            Block: min_example
                Block: calc_min
Building modules...
    Building module "min_example"...
        Constructing wrapper function "unknown_subroutine"...
          unknown_subroutine()
        Constructing wrapper function "unknown_subroutine"...
          unknown_subroutine()
        Constructing wrapper function "polint"...
          polint(xa,ya,x,y,dy,[n])
        Constructing F90 module support for "min_example"...
            Constructing wrapper function "min_example.calc_min"...
              calc_min()
    Wrote C/API module "min_example" to file "/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c"
    Fortran 90 wrappers are saved to "/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_example-f2pywrappers2.f90"
  adding '/tmp/tmpQULhjN/src.linux-x86_64-2.7/fortranobject.c' to sources.
  adding '/tmp/tmpQULhjN/src.linux-x86_64-2.7' to include_dirs.
copying /home/anasal/anaconda2/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.c -> /tmp/tmpQULhjN/src.linux-x86_64-2.7
copying /home/anasal/anaconda2/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.h -> /tmp/tmpQULhjN/src.linux-x86_64-2.7
  adding '/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_example-f2pywrappers2.f90' to sources.
build_src: building npy-pkg config files
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
customize Gnu95FCompiler
Found executable /usr/bin/gfortran
customize Gnu95FCompiler
customize Gnu95FCompiler using build_ext
building 'min_example' extension
compiling C sources
C compiler: gcc -pthread -B /home/anasal/anaconda2/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

creating /tmp/tmpQULhjN/tmp
creating /tmp/tmpQULhjN/tmp/tmpQULhjN
creating /tmp/tmpQULhjN/tmp/tmpQULhjN/src.linux-x86_64-2.7
compile options: '-I/tmp/tmpQULhjN/src.linux-x86_64-2.7 -I/home/anasal/anaconda2/lib/python2.7/site-packages/numpy/core/include -I/home/anasal/anaconda2/include/python2.7 -c'
gcc: /tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c
In file included from /home/anasal/anaconda2/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1809:0,
                 from /home/anasal/anaconda2/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
                 from /home/anasal/anaconda2/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from /tmp/tmpQULhjN/src.linux-x86_64-2.7/fortranobject.h:13,
                 from /tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:19:
/home/anasal/anaconda2/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it by " \
  ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:288:13: error: redefinition of ‘doc_f2py_rout_min_example_unknown_subroutine’
 static char doc_f2py_rout_min_example_unknown_subroutine[] = "\
             ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:228:13: note: previous definition of ‘doc_f2py_rout_min_example_unknown_subroutine’ was here
 static char doc_f2py_rout_min_example_unknown_subroutine[] = "\
             ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:292:18: error: redefinition of ‘f2py_rout_min_example_unknown_subroutine’
 static PyObject *f2py_rout_min_example_unknown_subroutine(const PyObject *capi_self,
                  ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:232:18: note: previous definition of ‘f2py_rout_min_example_unknown_subroutine’ was here
 static PyObject *f2py_rout_min_example_unknown_subroutine(const PyObject *capi_self,
                  ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:112:12: warning: ‘f2py_size’ defined but not used [-Wunused-function]
 static int f2py_size(PyArrayObject* var, ...)
            ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:228:13: warning: ‘doc_f2py_rout_min_example_unknown_subroutine’ defined but not used [-Wunused-variable]
 static char doc_f2py_rout_min_example_unknown_subroutine[] = "\
             ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:232:18: warning: ‘f2py_rout_min_example_unknown_subroutine’ defined but not used [-Wunused-function]
 static PyObject *f2py_rout_min_example_unknown_subroutine(const PyObject *capi_self,
                  ^
In file included from /home/anasal/anaconda2/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1809:0,
                 from /home/anasal/anaconda2/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
                 from /home/anasal/anaconda2/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from /tmp/tmpQULhjN/src.linux-x86_64-2.7/fortranobject.h:13,
                 from /tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:19:
/home/anasal/anaconda2/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it by " \
  ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:288:13: error: redefinition of ‘doc_f2py_rout_min_example_unknown_subroutine’
 static char doc_f2py_rout_min_example_unknown_subroutine[] = "\
             ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:228:13: note: previous definition of ‘doc_f2py_rout_min_example_unknown_subroutine’ was here
 static char doc_f2py_rout_min_example_unknown_subroutine[] = "\
             ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:292:18: error: redefinition of ‘f2py_rout_min_example_unknown_subroutine’
 static PyObject *f2py_rout_min_example_unknown_subroutine(const PyObject *capi_self,
                  ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:232:18: note: previous definition of ‘f2py_rout_min_example_unknown_subroutine’ was here
 static PyObject *f2py_rout_min_example_unknown_subroutine(const PyObject *capi_self,
                  ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:112:12: warning: ‘f2py_size’ defined but not used [-Wunused-function]
 static int f2py_size(PyArrayObject* var, ...)
            ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:228:13: warning: ‘doc_f2py_rout_min_example_unknown_subroutine’ defined but not used [-Wunused-variable]
 static char doc_f2py_rout_min_example_unknown_subroutine[] = "\
             ^
/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c:232:18: warning: ‘f2py_rout_min_example_unknown_subroutine’ defined but not used [-Wunused-function]
 static PyObject *f2py_rout_min_example_unknown_subroutine(const PyObject *capi_self,
                  ^
error: Command "gcc -pthread -B /home/anasal/anaconda2/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/tmp/tmpQULhjN/src.linux-x86_64-2.7 -I/home/anasal/anaconda2/lib/python2.7/site-packages/numpy/core/include -I/home/anasal/anaconda2/include/python2.7 -c /tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.c -o /tmp/tmpQULhjN/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.o -MMD -MF /tmp/tmpQULhjN/tmp/tmpQULhjN/src.linux-x86_64-2.7/min_examplemodule.o.d" failed with exit status 1

And I compile it using:

gfortran -c min_example.f90
gfortran -c qromb.f
f2py -c qromb.f min_example.f90 -m min_example

Any help would be very much appreciated! Thank you!

Update: I have continued searching around and have found very few leads. Someone suggested that f2py has an issue with inline comments, but removing them did not help. Another answer I read suggested converting the '.f' file to a '.f90' file, but to be honest, I don't know the difference between the two (I am very new to fortran). So I don't know how to go about it (the structure of the .f file is not so clear to me) and I don't know if it would slow it down.

Another update: Compiling using 'f2py -m min_example min_example.f90 qromb.f' gives the following output:

Reading fortran codes...
    Reading file 'min_example.f90' (format:free)
    Reading file 'qromb.f' (format:fix,strict)
Post-processing...
    Block: min_example
            Block: min_example
                Block: calc_min
            Block: qromb
            Block: trapzd
                    Block: func
            Block: polint
Post-processing (stage 2)...
    Block: min_example
        Block: unknown_interface
            Block: min_example
                Block: calc_min
            Block: qromb
            Block: trapzd
            Block: polint
Building modules...
    Constructing call-back function "cb_func_in_trapzd__user__routines"
      def func(x,q): return sum
    Building module "min_example"...
        Constructing wrapper function "qromb"...
routsign2map: Confused: function qromb has externals ['func'] but no "use" statement.
sign2map: Confused: external func is not in lcb_map[].
append_needs: unknown need 'func'
append_needs: unknown need 'func'
          qromb(func,a,b,ss,q,[func_extra_args])
        Constructing wrapper function "trapzd"...
          trapzd(func,a,b,s,n,q,[func_extra_args])
        Constructing wrapper function "polint"...
          polint(xa,ya,x,y,dy,[n])
        Constructing F90 module support for "min_example"...
            Constructing wrapper function "min_example.calc_min"...
              calc_min()
    Wrote C/API module "min_example" to file "./min_examplemodule.c"
    Fortran 90 wrappers are saved to "./min_example-f2pywrappers2.f90"

and generates the following files:

min_example-f2pywrappers2.f90
min_example.mod
min_example.o
qromb.o
min_examplemodule.c 

This looks promising but when I go into python, I can't import the function.

Answer 1

In order for F2PY to create wrappers for a Fortran procedure, it needs to fully identify the type and intent of the Fortran procedure arguments, i.e. is the argument an integer/real, scalar/vector, input/output/both, etc. This Fortran interface is referred to as the procedure's signature in F2PY. If a function is passed as an argument (e.g. func in trapzd and qromb in the linked file qromb.f), F2PY also needs to identify this information for that passed function's arguments. Unlike e.g. Fortran 90 and later versions, Fortran 77 (the dialect in which qromb.f seems to be written) does not provide any language to explicitly define this information directly in the Fortran code.

Thus, what happened in your case, is that F2PY failed to automatically identify procedure signatures with the information at hand (hence the unknown_subroutine output).

There are, however, a number of ways in which F2PY can successfully wrap your code:

1. Assist F2PY in identifying procedure signatures, by creating and manually modifying the procedure signature file for qromb.f.
2. Define the interface and make use of e.g. qromb in your min_example module, and pass your Fortran 77 code as a pre-compiled object to F2PY. See e.g. the first part of this answer.
3. Rewrite qromb.f in a more modern Fortran dialect, explicitly defining all interfaces (your code seems to be modified from Numerical Recipes in Fortran 77 for which a Fortran 90 version exists, that may be helpful).

You mention that you are a Fortran novice, and may therefore not want to modify your source code directly. With that in mind, I will describe the first of the listed solutions above in more detail as applied to your example:

The first step is to create a signature file (see The smart way in F2PY documentation), by executing the following command:

f2py -m min_example -h min_example.pyf min_example.f90 qromb.f

This creates a signature file called min_example.pyf, opening this file you will notice that the Fortran interfaces (F2PY signatures) to trapzd and qromb only appear as unknown_subroutine. In addition, the interface to polint requires modification.

Your second step is then to modify min_example.pyf, so that it consists of only the following text (also refer to the F2Py documentation on call-back functions):

!    -*- f90 -*-
! Note: the context of this file is case sensitive.

python module __user__routines 
    interface
        function fun(x,q) result(res) ! in :min_example:qromb.f
            real*8 intent(in)   :: x, q
            real*8              :: res
        end function fun
    end interface
end python module __user__routines

python module min_example ! in 
    interface  ! in :min_example
        module min_example ! in :min_example:min_example.f90
            subroutine calc_min ! in :min_example:min_example.f90:min_example
            end subroutine calc_min
        end module min_example
        subroutine qromb(func,a,b,ss,q) ! in :min_example:qromb.f
            use __user__routines, func=>fun
            external func
            real*8 intent(in)    :: a, b, q
            real*8 intent(out)   :: ss
        end subroutine qromb
        subroutine trapzd(func,a,b,s,n,q) ! in :min_example:qromb.f
            use __user__routines, func=>fun
            external func
            real*8 intent(in)       :: a, b, q
            integer intent(in)      :: n
            real*8 intent(inout)    :: s
        end subroutine trapzd
        subroutine polint(xa,ya,n,x,y,dy) ! in :min_example:qromb.f
            real*8 dimension(n),intent(in)                              :: xa
            real*8 dimension(n),intent(in),depend(n)                    :: ya
            integer, optional,intent(in),check(len(xa)>=n),depend(xa)   :: n=len(xa)
            real*8 intent(in)                                           :: x
            real*8 intent(out)                                          :: y, dy
        end subroutine polint
    end interface 
end python module min_example

! This file was auto-generated with f2py (version:2).
! See http://cens.ioc.ee/projects/f2py2e/

The third and final step is to compile your source code using F2PY and the manually corrected signature file, by executing the following command:

f2py -c min_example.pyf min_example.f90 qromb.f

Your example does not show how you intend to use the Fortran code in Python, but here is an example Python script, showing the use of the just compiled module:

import numpy as np
from scipy.integrate import romberg
import min_example

print(min_example.__doc__)
print(min_example.qromb.__doc__)

def func(x, q):
    return q*np.sin(x)

a = 0.0
b = np.pi
q = 1.0

f_f2py = min_example.qromb(func, a, b, q)
f_scipy = romberg(func, a, b, args=(q,))

print("f2py:  {:0.7g}".format(f_f2py))
print("scipy: {:0.7g}".format(f_scipy))

Which gives me the following output:

<auto-generated documentation strings>

f2py:  2
scipy: 2