Gnuplot Script for average over multiple files

Question

I have series of measurement in several files. Every file is looking like this:

1 151.973938 144.745789 152.21991 17:57:14
2 151.995697 144.755737 152.21991 17:57:14
3 152.015747 144.765076 152.21991 17:57:14
.
.
.

I'm looking for a possibility to compute the average of the same field in several files. At the end of the process I would like to have a graph of the average measurement.

Is that possible with gnuplot? I wasn't able to find a fitting option in gnuplot by myself. If not, which different way to achieve that would you recommend?

Best regards, Juuro

should those semicolons be there? – andyras May 30 '12 at 14:49 — andyras, May 30 '12 at 14:49
Uh, no! I deleted them. Thanks! – Juuro May 30 '12 at 16:39 — Juuro, May 30 '12 at 16:39

andyras · Accepted Answer · 2012-05-30T21:27:19.510

4

You cannot do it all in gnuplot. Gnuplot is limited to processing columns from one file at a time. You need some other utility to preprocess your data. Assuming the data is in the format you demonstrated (with spaces instead of semicolons), this script will take the average of the second, third and fourth columns and spit out a file with the first and fifth columns the same, and the middles averaged. Run this bash script in a directory where there are only the .txt files you want to process.

#!/bin/bash

sum=$(ls -l *.txt | wc -l)
paste -d" " *.txt | nawk -v s="$sum" '{
    for(i=0;i<=s-1;i++)
    {
        t1 = 2+(i*5)
        temp1 = temp1 + $t1
        t2 = 3+(i*5)
        temp2 = temp2 + $t2
        t3 = 4+(i*5)
        temp3 = temp3 + $t3
    }
    print $1" "temp1/s" "temp2/s" "temp3/s" "$5
    temp1=0
    temp2=0
    temp3=0
}'

From inside gnuplot, you can run the script like this:

!myscript.sh > output.out
plot 'output.out' u 1:2 # orwhatever

or like this:

plot '<myscript.sh' u 1:2

(code inspired by what I found here.)

edited May 30 '12 at 21:27

answered May 30 '12 at 15:03

andyras

15,542
6
55
77

You might be able to do it in gnuplot 4.6 using the `stats` command -- but the pain you'd have to go through to make it work probably isn't worth it. – mgilson May 30 '12 at 17:41
Also, just looking at your last line, I think it should just be `plot ' – mgilson May 30 '12 at 20:13
@mgilson: Actually I think it's *not* possible with stats, because `stats` computes *one* mean value for a column. So you can compute the mean values of a column ("vertical mean"), but not the mean value across two or more files ("horizontal mean"). [gnuplot 5.2] – U. Windl Apr 08 '19 at 20:56

Raphael Roth · Answer 2 · 2012-05-30T14:53:24.660

I think it's no possible with gnuplot. I would first make a script which does the averaging and prints the results to stdout. Say this script is called average.py:

plot '<average.py FILE1 FILE2 FILE3' w l

the script average.py can look like this for example.

#!/usr/bin/python
from numpy import loadtxt,mean,ones
import sys 

#number of files:
nrfiles=len(sys.argv[1:])

#just to get the dimensions of the files
data=loadtxt(str(sys.argv[1]))
rows=data.shape[0]
cols=data.shape[1]

#initialize array
all=ones((int(nrfiles),int(rows),int(10)))

#load all files:
n=0
for file in sys.argv[1:]:
      data=loadtxt(str(file))
      all[n,:,0:cols]=data
      n=n+1

#calculate mean:
mean_all=mean(all,axis=0)

#print to stdout
for i in range(rows):
      a=''
      for j in range(cols):
         a=a+str('%010.5f   ' % mean_all[i,j])
      print str(a)

The limits of this script is that all files must have the same data-structure

You might be able to do it in gnuplot 4.6 using the `stats` command -- but the pain you'd have to go through to make it work probably isn't worth it — mgilson, May 30 '12 at 17:41

Gnuplot Script for average over multiple files

2 Answers2

Linked