Mean Value of Multiple Files?

Question

I'm new to Fortran95 and I've got the following task:

I've got N files with research data named as: file0001.asc, file0002.asc, ..., fileN.asc. Each of these files contains I x J numbers (equal for all files) and what I need to do is write a new file (let's name it output.asc) with I x J numbers each of them being the mean (average) value of the same element of N files. For example element:

output(1,3) = sum(file0001(1,3),file0002(1,3),...,fileN(1.3)) / N

The whole process needs to be pretty automated, so the program should count the number of files N in the folder and also the dimension of the elements of the files I x J because I will need to do it for many different sets of data.

EDIT:

So far I've written some (pseudo) code, looking like this:

PROGRAM MultiTableMeanValue 
DIMENSION out(?,?)  !need to find this dimension somehow

OPEN(4,file='output.asc',form='formatted')
10 FORMAT('iso',i3.3,'.dat')

!I create the matrix to sum up the values of the output matrix in every loop

DO i=1,?             ! (Where “?” is the number of rows and columns which is need to find somehow
 DO j=1,?            ! since I can’t do it manually cause they are like 1000+).
  S(I,j)=0
 ENDDO
ENDDO

READ*, N  !N is the number of files

DO i=001,N !somehow need its format to be 001,002 cuz N is around 400-900, 
           !so I’ll need ‘10 FORMAT’ above
 OPEN(5,file='file',N,'.asc') !My files are file001,file002,…,fileN and 
                              !I need to open each one of these in every loop.
                              !Write their numbers in “out” matrix
                              !then SUM with each of the other files in every
                              !loop and then divide by N.
 DO j=1,?          ! (Where “?” is again the unknown number of rows and columns
  DO k=1,?            
   READ(5,*) out(j,k)
   S(j,k)=S(j,k)+out(j,k)
  END DO
 END DO 
END DO

DO i=1,?
 DO i=1,?
  S(i,j)=S(i,j)/N
  WRITE(4,*)  S(i,j)
 ENDDO
ENDDO

CLOSE(4)
CLOSE(5)

END PROGRAM MultiTableMeanValue

Question:

How can I expand this initial idea to complete my task?

With which particular part of this are you struggling? We can address problems you have, but few will write a program to your specifications from scratch. — francescalus, Sep 23 '15 at 12:57
Well, first you may need to list the files in the directory. Have a look at http://stackoverflow.com/questions/13601967/printing-directory-listing-in-fortran Or you may try to open them starting from 1 until you get an error because there are no more files (assuming that there are no holes). — Edmondo Giovannozzi, Sep 23 '15 at 14:38
Look at http://stackoverflow.com/questions/1262695/converting-integers-to-strings-in-fortran and use it to get the filename. — Vladimir F Героям слава, Sep 25 '15 at 12:57
Always use `implicit none` in your Fortran programs, in this case insert that statement right after the `PROGRAM` statement. And invest in some space characters to indent the lines in your code to reflect its structure -- they're a great aid to matching up `do` and `end do` and similar constructs. — High Performance Mark, Sep 25 '15 at 13:37
Also, I'm not sure you need fortran**95**. And anyway, tags belong with the tags, not in the question title. — Andras Deak -- Слава Україні, Sep 25 '15 at 13:55
AndrasDeak, ok removed it. HighPerformanceMark, Okey i'll have that in mind but now I'm fine with i,j,..,n as integers. Of course, in my program, I always use space characters in DO, IF etc constracts. Anyway I'll add it here too. — Crusifizer, Sep 25 '15 at 14:52

roygvib · Answer 1 · 2015-09-26T19:25:02.313

The missing part in your program is how to get the number of rows and columns in each file. This can be done, for example, by using wc command:

integer :: ncol, nrow, nfile
call system( "head -n 1 file001.dat | wc -w > count ; wc -l file001.dat >> count ; ls file*.dat | wc -w >> count" )

open( 10, file="count", status="old" )
read( 10, * ) ncol, nrow
read( 10, * ) nfile
close( 10 )

where

head -n 1 file001.dat | wc -w counts the number of columns in the first line (ncol)
wc -l file001.dat counts the number of rows in a file (nrow)
ls file*.dat | wc -w counts the number of files (nfile)

These numbers are saved in a temprary file and read into the program. Necessary arrays are then allocated as

real, allocatable, dimension(:,:) :: inp, out
allocate( inp( nrow, ncol ), out( nrow, ncol ) )

The remaining part is almost the same as OP's program:

character(100) :: str
integer :: ifile

out = 0.0
do ifile = 1, nfile
    write( str, "('file', i3.3, '.dat')" ) ifile    !! set file names like file007.dat
    open( 10, file=trim(str), status="old" )
    do irow = 1, nrow
        read( 10, * ) inp( irow, : )
    enddo
    close( 10 )
    out = out + inp
enddo
out = out / real( nfile )

open( 10, file="output.dat" )
write( str, "('(', i0, 'f15.6)')" ) ncol
do irow = 1, nrow
    write( 10, str ) out( irow, : )
enddo
close( 10 )

To retain the accuracy, it may be better to use double precision when the input data are very different in magnitude or having alternate signs. Also, if your compiler is sufficiently new, you can use execute_command_line instead of system.

Are you sure OP is running on Linux? If not, the `wc` is probably not available. A more general solution would `read(10, '(a)', advance = 'no', iostat = io)` to read a line one character at a time until `io /= 0` and count the columns. A similar procedure could be used to count the rows. — Jeff Irwin, Sep 26 '15 at 21:19
@JeffIrwin I have assumed that OP was using Linux, but this may not be the case. If so, it is straightforward to write a code to get nrow and ncol (e.g., http://stackoverflow.com/questions/32022062/reading-a-sequence-of-integer-in-a-line-with-unknown-bound-in-fortran/32023397#32023397) but I don't know how to count the number of files in Windows... — roygvib, Sep 26 '15 at 21:39
The number of files could be read as user input, or one could loop over files until no more files are available (as suggested in the comments in the question). Hope OP is using Linux :) — roygvib, Sep 26 '15 at 21:47

Mean Value of Multiple Files?

EDIT:

Question:

1 Answers1