2

I have a fortran code which produces unformatted files. I wrote a script in python to plot some quantities, and I use the numpy fromfile routine to read the data from the file. In my laptop, which is a MacBook pro with OSX 10.9.1 mavericks, the data are read fine. The piece of the code where it reads the data is as follows:

fileid = open(file,'rb')
hdel = np.fromfile(fileid,dtype='float32',count=1)
probtmp = np.fromfile(fileid,dtype='float64',count=maxptsr*maxptsz)

Now, I want to use it in another machine, a AMD x86_64 linux machine. This machine is x86_64, so I assume that is little-endian as well.

When I read the data I get different numbers. For example, for one value in my laptop I get 2.3892744070368817e-32, while in the AMD machine I get 2.6284548901535996e-34.

Does anyone know why I get different numbers? Thank you.

Alexander Vogt
  • 17,879
  • 13
  • 52
  • 68
AlexNoir
  • 789
  • 3
  • 8
  • 20
  • without any code, it's rather difficult to say what the problem might be... – MattDMo Jan 14 '14 at 19:59
  • As MattDMo said, it's very difficult to say what the problem might be without some idea of exactly what you're doing. What does the `fromfile` command look like? There are lots of non-endian things that may change between platforms (particularly with regards to extended precision floats - e.g. `np.longdouble`/`np.float128`/`np.float96`/etc). – Joe Kington Jan 14 '14 at 20:06
  • Sorry guys. I have edited the question. – AlexNoir Jan 14 '14 at 20:08
  • have you explicitly experimented with endianness? You can specify endian-specific dtypes as dtype='f8' – Eelco Hoogendoorn Jan 14 '14 at 20:18
  • @EelcoHoogendoorn - Beat me to it! @AlexNoir - It would also be good to verify that the file content is the same on both systems. Both should be little endian, and swapping endian-ness wouldn't give you the numbers you're seeing, regardless. The `fromfile` statement looks fine. I'd wager that the problem is deeper in your fortran code. It's definitely worth a quick check to see whether the file you're trying to read is actually the same on both systems. – Joe Kington Jan 14 '14 at 20:21
  • Is either of those numbers known to be correct? Take a look at some of the data *before* you write it to the file, so you'll know what to expect when you read it. – Warren Weckesser Jan 14 '14 at 20:45
  • @AlexNoir - Just as one final thought, you may (probably?) just be seeing the effects of numerical instability. Different LAPACK libraries implement operations in different ways. Very large or small numbers (or ill-conditioned matrices, etc), will magnify these differences. Furthermore, you should be aware that many common scientific operations (e.g. finding the eigenvalues/eigenvectors of a matrix) don't have a unique solution. Just because something like `eig` gives a different answer on different systems doesn't mean that it's incorrect. – Joe Kington Jan 14 '14 at 20:46
  • 2
    Exactly what are the Fortran statements that created the file? Binary unformatted Fortran output created using sequential acces contains record length information (http://stackoverflow.com/questions/8751185/fortran-unformatted-file-format, or google for many more links). You'll have to take that into account when you read the data with numpy. – Warren Weckesser Jan 14 '14 at 20:49
  • 1
    You could also try this: http://wiki.scipy.org/Cookbook/FortranIO/FortranFile – Warren Weckesser Jan 14 '14 at 21:00
  • Thank you all for the comments. @EelcoHoogendoorn, I have changed the type to big-endian, and the results are really noisy. – AlexNoir Jan 14 '14 at 23:19
  • @JoeKington, I know that the data files are the same because I replaced them by the original ones. The point is that in one computer the script gives the right plots and the same script gives noisy plot in another computer. The data are the results from a partial differential equation, and it is to much error (the second decimal, and more important, the order of the magnitude) to be caused only by the architecture of the machines. Thank you for the help anyway! – AlexNoir Jan 14 '14 at 23:25
  • @WarrenWeckesser, I write the files using direct access, so only one delimiter is expected at the beginning of the file (and another in the end, of course!). – AlexNoir Jan 14 '14 at 23:28

0 Answers0