MPI: writing and read integers to a file

Question

Just started off with Openmpi. Trying to write and read ints to a file .. code for writing:

the characters getting written to the file are unrecognizable, mostly garbage.

#include <stdlib.h>
#include <stdio.h>
#include "mpi.h"

#define BUFSIZE 10
#define FIRSTCHAR 1
#define FILENAME "file1.dat"

int main(int argc, char* argv[]) {
  int i, np, me;
  int buf[BUFSIZE];     /* The buffer to write */
  MPI_File myfile;       /* Shared file */

  /* Initialize MPI */
  MPI_Init(&argc, &argv);
  MPI_Comm_rank(MPI_COMM_WORLD, &me);
  MPI_Comm_size(MPI_COMM_WORLD, &np);

  /* Initialize buf with characters. Process 0 uses 'a', process 1 'b', etc. */
  for (i=0; i<BUFSIZE; i++) {
    buf[i] = FIRSTCHAR+(me);
  }

  /* Open the file */
  MPI_File_open (MPI_COMM_WORLD, FILENAME, MPI_MODE_CREATE | MPI_MODE_WRONLY,
         MPI_INFO_NULL, &myfile);
  /* Set the file view */
  MPI_File_set_view(myfile, me*BUFSIZE*sizeof(int), MPI_INT, MPI_INT, 
            "native", MPI_INFO_NULL);
  /* Write buf to the file */
  MPI_File_write(myfile, buf, BUFSIZE*sizeof(int), MPI_INT, MPI_STATUS_IGNORE);
  /* Close the file */
  MPI_File_close(&myfile);
  MPI_Finalize();
  exit(0);
}

dosen't work .. Please help!

changing paramenters of the api MPI_File_write(myfile, buf, BUFSIZE, MPI_INT, MPI_STATUS_IGNORE); dosent help .. let it be noted — Anirudh, Jul 06 '12 at 03:26

score 6 · Accepted Answer · answered Jul 06 '12 at 05:09

6

The files written are not garbage they are just in binary

if your on linux

first backup your file

cp file file2

then

try this command to convert from binary to ascii

hexdump -v -e '7/4 "%10d "' -e '"\n"' file2

also a side note: unless your writing millions of lines to file, you might not even need the mpi_file write /load functions

answered Jul 06 '12 at 05:09

pyCthon

11,746
20
73
135

1

hi!! ur brilliant :D works , thanks a lot, also, yeah will be writing a few million lines so using mpi apis – Anirudh Jul 06 '12 at 05:38
could u help me with reading , im getting a runtime error, it does read successfully though, from the file – Anirudh Jul 06 '12 at 05:55
you could also just do this http://stackoverflow.com/questions/9777828/writing-a-matrix-into-a-single-txt-file-with-mpi – pyCthon Jul 06 '12 at 06:08

unkulunkulu · Answer 2 · 2012-07-06T13:03:10.847

1

The ints are being written to the file in binary mode, i.e. they're not converted to strings and printed like you do screen output using printf, instead their memory representation is copied to the file somehow. If you write code to read them from the file (using MPI_File_read), it will work properly.

Note that that binary format can be quite tricky to read by yourself, because MPI tries to be cross-platform and probably writes some additional type information, so you should use MPI to read from those files.

edited Jul 06 '12 at 13:03

answered Jul 06 '12 at 03:32

unkulunkulu

11,576
2
31
49

i c, so by writing using mpi api's, i wil be able to read back only if i use mpi_read ?? correct? – Anirudh Jul 06 '12 at 05:31
In practice, the output from MPI_File_write is just binary; you could read in all np*BUFSIZE characters with `fread(buf, sizeof(char), np*BUFSIZE, fileh);`, for instance. – Jonathan Dursi Jul 06 '12 at 12:50

Aeternus · Answer 3 · 2019-03-19T06:11:46.007

1

Here is a more concise command to view your binary file in ascii On Linux: Backup your file, as pyCthon suggested:

cp file temp

Then, run the following command to print out file as ascii characters:

od -c temp

If your data happens to be integers:

od -i temp

edited Mar 19 '19 at 06:11

answered Mar 19 '19 at 05:52

Aeternus

346
4
15

MPI: writing and read integers to a file

3 Answers3