Python unformatted save

Question

I am using python to create the input for a program. This program takes an unformatted binary file as input. If I were using fortran I would create this file with

   open (10,file=outfile,status='unknown',form='unformatted')
   write(10) int1,int2,int3,int4,list0
   write (10) list1
   write (10) list2
   write (10) list3
   write (10) list4
   close (10)

Is there a way to create the same kind of file in python? My first guess would be to create a subroutine in fortran which can save files given some inputs and then implement this in my python code using f2py, but I don't really know how one would go about doing this. The lists that I am writing to file are very large and the exact structure is very important. This means that answers such as Writing Fortran unformatted files with Python seem to be unsatisfactory as they don't adequately deal with headers in the file/endianess and so on.

In my python code a have a 2-d array, each row containing the x,y,z coordinates and the mass of the particle. This data needs to be split among a number of files.

For the particle load the structure of the files is:

BLOCK-1 - body is 48 bytes long:

  nparticles_this_file   -    integer*4   (nlist)
  nparticles_total       -    integer*8
  number of this file    -    integer*4
  total number of files -     integer*4
  Not used              -   7*integer*4

BLOCK-2

 A list of nlist  x-coordinates   (real*8)

(the x-coordinate is in units of the periodic box size 0<=x<1)

BLOCK-3

 A list of nlist  y-coordinates   (real*8)

(the y-coordinate is in units of the periodic box size 0<=y<1)

BLOCK-4

 A list of nlist  z-coordinates   (real*8)

(the z-coordinate is in units of the periodic box size 0<=z<1)

BLOCK-5

A list of nlist particle masses  (real*4)

in units of the total mass in the periodic volume

Have you had a look at [struct](https://docs.python.org/3/library/struct.html) and [array](https://docs.python.org/3/library/array.html)? These modules don't care about headers but may help when starting from scratch. (There are also libraries for almost everything on [pypi](https://pypi.python.org/pypi) ) — janbrohl, Aug 26 '16 at 15:21
The header and structures of each section have to follow a very specific format, reading the documentation for these there doesn't seem to be a simple way to achieve the desired structure — Jack, Aug 26 '16 at 15:53
What is the structure that you are talking about? I believe that if you show a minimal structure, someone will come up with a helpful tip that you can go from. — innoSPG, Aug 26 '16 at 16:08
Can you post the fortran code used to read the file? Can you explain more clearly whats wrong with the answer you linked? — agentp, Aug 26 '16 at 19:00

score 2 · Accepted Answer · answered Aug 26 '16 at 17:43

A code like the following shall be a good starting point for what you are trying to do. The structure of your data is not complicated as what I expected to be from your explanation. I wrote a small function to write on list as it is pretty repetitive. The most important thing to notice is that fortran unformatted file write the size of each record along with the record (before and after the record). That helps fortran itself to check for basic error when reading the file later. Using fortran stream files will spare you from writing the record size.

import numpy as np

def writeBloc(dList, fId):
    """Write a single list of data values ad float 64 or fortran real*8"""
    np.array([len(dList)*8],np.int32).tofile(fId) # record size
    np.array([dList],np.float64).tofile(fId)
    np.array([len(dList)*8],np.int32).tofile(fId) # record size


int1,int2,int3,int4 = 4, 100, 25, 25
#
f = open("python.dat", "wb")
# Block 1
np.array([48],np.int32).tofile(f) # record size
np.array([int1],np.int32).tofile(f)
np.array([int2],np.int64).tofile(f)
np.array([int3],np.int32).tofile(f)
np.array([int4],np.int32).tofile(f)
np.zeros((7),np.int32).tofile(f) # list0
np.array([48],np.int32).tofile(f) # record size
#
list1=[10.0, 11.0, 12.0, 13.0]
list2=[20.0, 21.0, 22.0, 23.0]
list3=[30.0, 31.0, 32.0, 33.0]
list4=[40.0, 41.0, 42.0, 43.0]
# data
writeBloc(list1, f) # Block 1
writeBloc(list2, f) # Block 2
writeBloc(list3, f) # Block 3
writeBloc(list4, f) # Block 4
f.close()

Python unformatted save

1 Answers1