Read Fortran binary into Python

Question

I have a Fortran file that generates some numbers in a two-dimensional array, for example:

program WriteBinaryFile

    real(kind=4)::myarray(140,130)
    integer(kind=4):: num1, num2
    character*120 filename
    
    ! Insert some numbers, i.e., number 1 through 18200=140*130
    do num=1,140
        do num2=1,130
           myarray(num1,num2)=(num1-1)*130+num2
        end do
    end do

    filename="binaryFile"
        
    open(22,file=filename,form='unformatted',access='direct',recl=140*130*4,&
         iostat=ios)

    irec=1
    write(22,rec=irec)myarray

    close(22)

    ! Some other things happen.

    end program WriteBinaryFile

I am trying to read this into Python, but so far I have had no success at all; with either numpy.fromfile or scypi.io.FortranFile:

import numpy as np
f = np.fromfile("binaryFile")

or

from scypi.io import FortranFile
f = FortranFile("binaryFile", "r")
myarray = f.read_reals()

They don't work. The first program outputs a bunch of numbers that don't make sense and the latter throws an exception that states: "scipy.io._fortran.FortranFormattingError: End of file in the middle of a record".

I have tried to do different things, as outlined in the posts:

1. Reading Fortran binary file in Python
2. https://numpy.org/doc/stable/reference/generated/numpy.fromfile.html
3. https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.FortranFile.html
4. etc.

to no avail! With the first link being the closest. Help solving this issue would be appreciated.

I have been able to recover the file content using Fortran, but not Python.

Other information.

• This is a transcript of the program from my machine. The original file was generated in a machine of which I have no knowledge about, other than it generates the file; i.e., no idea of encoding, compiler version, etc..

Answer 1

Firstly, your Fortran example is incorrect. It creates some myarray, but then writes some toast. Always use IMPLICIT NONE in any Fortran code you share with others.

scipy.io.FortranFile will be useless for this case. It is for sequential access files. Your file is direct access. In practice, the direct access file will be the same as a stream access file and the same as a file written in C (but in column-major order).

You can use numpy.fromfile to read the data

 array_data = np.fromfile(file, dtype=np.float32).reshape([nx, ny], order='F')

If the machine used to create the file was some bigendian-RISC, you may have to convert the byte order (ndarray.byteswap or by reading the buffer as bigendian directly).

I suggest opening the file in a hex editor and see whether the numbers stored there make sense as littleendian floats or as bigendian floats.

Answer 2

The docs (https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.FortranFile.html) say that you should use file extensions, as seen here:

from scipy.io import FortranFile
f = FortranFile('test.unf', 'w')
f.write_record(np.array([1,2,3,4,5], dtype=np.int32))
f.write_record(np.linspace(0,1,20).reshape((5,4)).T)
f.close()

Try writing it with the file extension. Hope this helps!