calculating rolling average using python

Question

I've just started learning python. I am using it to write a script to calculate salt inflow rolling average. I have data like that

Date    A4260502_Flow   A4261051_Flow   A4260502_EC A4261051_EC
25/02/1970  1304    0   411 0   1304            
26/02/1970  1331    0   391 0   1331            
27/02/1970  0   0   420 411 0           
28/02/1970  0   0   400 391 0           
1/03/1970   0   0   0   420 0           
2/03/1970   1351    1304    405 400 1327.5      
3/03/1970   2819    1331    415 405 2075        
4/03/1970   2816    0   413 0   2816            
5/03/1970   0   1351    0   415 1351            
6/03/1970   0   0   0   0   0           
7/03/1970   0   2819    0   413 2819            
8/03/1970   0   0   0   0   0           
9/03/1970   0   2816    0   412 2816

And my script is

inputfilename = "output.csv"
outputfilename = "SI_calculation.csv"

# Open files
infile = open(inputfilename,"r+")
outfile = open(outputfilename,'w')

# Initialise variables  
EC_conversion = 0.000525
rolling_avg = 5
flow_avg_list = []
SI_list = []
SI_ra_list = []
SI_ra1 = []

# Import module
import csv
import numpy                 #L20


table = []
reader = csv.reader(infile)         #read
for row in csv.reader(infile):
    table.append(row)
infile.close()

for r in range(1,len(table)):        
    for c in range(1,len(row)): #l30
        table[r][c] = float(table[r][c])


#Calculating flow average
for r in range(1,len(table)):

    flow1 = table[r][1]
    flow2 = table[r][2]
    if flow1 == 0.0:                 
        flow_avg = flow2            #l40
    elif flow2 == 0.0:
        flow_avg = flow1
    else:
        flow_avg = (flow1+flow2)/2
    flow_avg_list.append(flow_avg)

#Calculating salt inflow
for r in range(1,len(table)):
    s1 = table[r][3]                               
    s2 = table[r][4]        #l50
    if s1 == 0.0 or s2 == 0.0 or flow_avg_list[r-1] == 0.0:
        SI = 0.0
    else:
        SI = EC_conversion*flow_avg_list[r-1]*(s2-s1)
    SI_list.append(SI)
print SI_list    

#Calculating rolling average salt inflow
for r in range(1,len(table)):                 
    if r < 5:       #rolling-avg = 5
        for i in range(0,r+5):      #l60
            S = SI_list[i]
            SI_ra1.append(S)
        SI_ra = numpy.mean(SI_ra1)
        SI_ra_list.append(SI_ra)
    elif r > (len(table) - 4):
        for i in range(r-5,len(table)-1):
            S = SI_list[i]
            SI_ra1.append(S)
        SI_ra = numpy.mean(SI_ra1)
        SI_ra_list.append(SI_ra)    #l70
    else:
        for i in range(r-5,r+5):
            S = SI_list[i]       #Line 73
            SI_ra1.append(S)
        SI_ra = numpy.mean(SI_ra1)
        SI_ra_list.append(SI_ra)
print SI_ra_list

When I ran the script it gave me the error: Line 73: list index out of range. Does anyone know what the error could be? Sorry this is a long script. I don't know how to shorten it yet.

I've just marked line 73 with #Line73. This is in the last bit of the code. — Amylee, May 13 '13 at 02:24

score 3 · Answer 1 · edited May 23 '17 at 12:03

3

Please throwaway your code and start over using the answer to this question as a base: Rolling Average to calculate rainfall intensity

Not that your code couldn't work, but there is no need to write Fortan code in Python. The question that I linked to makes better use of Python features, and if you work through this, including following up the link to the question with the Interpolate class Linear Interpolation - Python then you will save yourself untold hours of struggling.

No need to make all the mistakes yourself. Start by imitating the masters, then customize their techniques to suit your needs, and in a few years, you too will be a master of Python.

edited May 23 '17 at 12:03

Community

1
1

answered May 13 '13 at 02:25

Michael Dillon

31,973
6
70
106

I agree with the principle, but I think a [pandas](http://pandas.pydata.org)-based solution would be both more powerful and easier than either of the linked answers. – DSM May 13 '13 at 02:50
I agree, but this is stack overflow where people come to learn how to write code. Once Amylee gets a few Python programs working, then the wonders of NumPy, PANDAS, HDF5 and so on, will be worth exploring. Everyone needs to start somewhere, but in 2013 using Python, that start should include list comprehensions, the with keyword, and so on. – Michael Dillon May 13 '13 at 03:52
Thanks all. I'll give it a try. It does take time as I'm also new to programming language not just python in particular. Only started it last week. – Amylee May 13 '13 at 04:35

score 2 · Answer 2 · answered May 13 '13 at 02:24

2

On line 65, change the condition to:

elif r > (len(table) - 5):

The problem is as you iterate towards the end of the list on line 73, you are trying to get the next 5 datapoints in the list, but there are only 4 datapoints left in the list, so you are indexing beyond the length of the list, hence the exception is thrown.

answered May 13 '13 at 02:24

qwwqwwq

6,999
2
26
49

That answer might be useful if you actually explained why you want to do that, and most importantly, what does that mean in the context of calculating a rolling average. – Michael Dillon May 13 '13 at 02:26

calculating rolling average using python

2 Answers2