Passing large amount of data from FORTRAN to C++

Question

What is the best and most efficient method to pass large amount of data (double numbers) from a FORTRAN program to a C++ program? right now I am using binary file but it is not fast enough!

I tried pipe. I followed http://msdn.microsoft.com/en-us/library/ms682499(VS.85).aspx for the C++ part. But for the FORTRAN part (which is the child processor) I do not know how to properly write data. Do I have to write on the console like WRITE(*,*) AllTheNumbers? because writing on the console takes a lot of time!

My FORTRAN code to write data:

        DO 281 I=1,NDOF 
        DO 280 J=1,UBW              
        IF (S(I,J).NE.0) THEN      
          WRITE (*, 2770) I,(J+I-1)
          WRITE (*,2760) (S(I,J))          
        ENDIF        
 280    CONTINUE         
 281    CONTINUE

http://msdn.microsoft.com/en-us/library/aa366551(v=vs.85).aspx — Mooing Duck, May 18 '13 at 01:00
see vladimir F's answer here http://stackoverflow.com/questions/13027067/write-unformatted-binary-data-to-stdout. — agentp, May 18 '13 at 12:04

M. S. B. · Accepted Answer · 2013-05-19T17:06:55.203

4

The fastest way would be to make it one mixed-language program. Generate the numbers in one (e.g., Fortran) and call the other (e.g., C++) from that language. With the ISO C Binding of Fortran, calling C is part of the Fortran language standard. Numerous Fortran compilers support this. Use "extern C" on the C++ side. Passing an array of doubles should be easy.

EDIT: if you continue using an IO method of transferring the information, you probably should transfer the information as binary data. You sample Fortran code is using formatted IO ... converting the internal binary representation of the numbers into human-readable characters is slow. In your Fortran open statement use: access='stream', form='unformatted'. Don't use a format on the write.

edited May 19 '13 at 17:06

answered May 18 '13 at 01:13

M. S. B.

28,968
2
46
73

A main C++ program which has the interface calls multiple FORTRAN programs (as a child processors) and each of them produces an array of doubles. I need to pass all the arrays to the main program. So I can not mixed them – VecTor May 18 '13 at 01:22
One would note if it is a multidimensional array, then fortran and c store arrays differently. Column/Row for fortran vs Row/column for c – EvilTeach May 18 '13 at 01:37
It seems possible to mix them: C++ calls Fortran subroutine sub1 to return array1, sub1 returns, then C++ calls Fortran sub2 to return array2, sub2 returns, etc. If you need even more speed, make the C++ program multi-threaded, have multiple arrays, and call the different Fortran subroutines from different threads. – M. S. B. May 18 '13 at 02:00
First it is easy to overcome, second it is not unheard of in C programs to se the Fortran ordering (C has no true dynamically allocated multidimensional arrays and offen usss custim indexing macros). – Vladimir F Героям слава May 18 '13 at 02:03
How about if I want to use it on the network? say for example fortran programs execute from different computers. can I still use mixed-language program? – VecTor May 18 '13 at 02:04

score 1 · Answer 2 · answered May 18 '13 at 01:38

Conside creating some shared memory. The c program 'creates it', and writes the data into it, and invokes the fortran child program. The fortran program 'maps'the shared memory in, and processes it and exits. The c program then destroys the memory and exits.

score 0 · Answer 3 · edited May 23 '17 at 11:43

A small variant on the answer over here: Write unformatted (binary data) to stdout, illustrating the repeated use of a small buffer.

  real(kind=8) :: x(256)    ! note real(kind=8) is "probably" an 8 byte float.
                            ! but not strictly guaranteed in fortran.
  character(len=2048)::  buffer  ! 2048 == 256*8
  do j=1,1000
     do i = 1,256
        x(i)=(i-1)**2+j*256
     enddo
     buffer=transfer(x,buffer)
     write(*,'(a)',advance='no')buffer
  enddo
  end

and python code to read it, just for example. (python struct module makes it nicer than c++ I think for debugging these things)

import subprocess
import struct
p=subprocess.Popen('./transfer',stdout=subprocess.PIPE)
a=" "
while a!="":
a=p.stdout.read(8)
if a!="" :  print struct.unpack('d',a)

I've never run into an issue with "unprintable characters" while writing like this. You can not use this trick to read from stdin in fortran however because of random strings of bits being interpreted as an EOF.

@VecTor, i just checked this works just fine over a network with Popen(['ssh', .. ] ). Of course you do need to worry about byte order issues in that case. — agentp, May 21 '13 at 20:19

Passing large amount of data from FORTRAN to C++

3 Answers3

Linked