Simple Moving Average summation/offset issue

Question

I wrote a simple moving average with a moving window of Temperatures read as a voltage between 0 and 10V.

The algorithm appears to work correctly, however, it has a problem where depending upon which Temperatures filled the window first, the moving average will have an offset for any values not near this value. For example, running this program with the temp. sensor plugged in a room temp yields 4.4V or 21.3 C. Though, if I unplug the temp. sensor the voltage drops to 1.4V yet the moving average stays at 1.6V. This offset becomes smaller as I increase the size of the window. How remove this offset even for small window sizes eg. 20 ?

REM SMA Num Must be greater than 1
#DEFINE SMANUM 20
PROGRAM
'Program 3 - Simple Moving Average Test
CLEAR
DIM SA(1)
DIM SA0(SMANUM) : REM Moving Average Window as Array
DIM LV1
DIM SV2
LV0 = 0 : REM Counter
SV0 = 0 : REM Average
SV1 = 0 : REM Sum
WHILE(1)
    SA0(LV0 MOD SMANUM) = PLPROBETEMP : REM add Temperature to head of window
    SV1 = SV1 + SA0(LV0 MOD SMANUM) : REM add new value to sum
    IF(LV0 >= (SMANUM)) : REM check if we have min num of values
        SV1 = SV1 - SA0((LV0+1) MOD SMANUM) : REM remove oldest value from sum
        SV0 = SV1 / SMANUM : REM calc moving average
        PRINT "Avg: " ; SV0 , " Converted: " ; SV0 * 21.875 - 75
    ENDIF
    LV0 = LV0 + 1 : REM increment counter
WEND
ENDP

(Note this is written in ACROBASIC for the ACR9000 by Parker)

Output - Temp Sensor attached

Raw: 4.43115    Avg: 4.41926     Converted: 21.6713125
Raw: 4.43115    Avg: 4.41938     Converted: 21.6739375
Raw: 4.43359    Avg: 4.41963     Converted: 21.67940625
Raw: 4.43359    Avg: 4.41987     Converted: 21.68465625
Raw: 4.43359    Avg: 4.42012     Converted: 21.690125
Raw: 4.43359    Avg: 4.42036     Converted: 21.695375
Raw: 4.43359    Avg: 4.42061     Converted: 21.70084375

...remove temp sensor while program is running

Raw: 1.40625    Avg: 1.55712     Converted: -40.938
Raw: 1.40381    Avg: 1.55700     Converted: -40.940625
Raw: 1.40625    Avg: 1.55699     Converted: -40.94084375
Raw: 1.40625    Avg: 1.55699     Converted: -40.94084375
Raw: 1.40381    Avg: 1.55686     Converted: -40.9436875
Raw: 1.40381    Avg: 1.55674     Converted: -40.9463125
Raw: 1.40625    Avg: 1.55661     Converted: -40.94915625

A noticeable offset appears between the raw and moving average after removing the sensor.

The offset also occurs in the reverse order:

Output - Begin program w/ Temp Sensor removed

Raw: 1.40381    Avg: 1.40550     Converted: -44.2546875
Raw: 1.40625    Avg: 1.40550     Converted: -44.2546875
Raw: 1.40625    Avg: 1.40549     Converted: -44.25490625
Raw: 1.40625    Avg: 1.40549     Converted: -44.25490625
Raw: 1.40625    Avg: 1.40548     Converted: -44.255125
Raw: 1.40625    Avg: 1.40548     Converted: -44.255125

... attach temp sensor while program is running

Raw: 4.43848    Avg: 4.28554     Converted: 18.7461875
Raw: 4.43848    Avg: 4.28554     Converted: 18.7461875
Raw: 4.43848    Avg: 4.28554     Converted: 18.7461875
Raw: 4.43848    Avg: 4.28554     Converted: 18.7461875
Raw: 4.43848    Avg: 4.28554     Converted: 18.7461875
Raw: 4.43359    Avg: 4.28530     Converted: 18.7409375

Again noticeable offset appears between the raw and moving average after attaching the sensor.

What has this to do with VBA? – Siddharth Rout Apr 22 '13 at 20:08 — Siddharth Rout, Apr 22 '13 at 20:08
Mistagged. Thanks I've updated it. – Ryan R Apr 22 '13 at 20:11 — Ryan R, Apr 22 '13 at 20:11

Simon · Accepted Answer · 2013-04-23T22:45:24.010

1

The problem seems to be that the value that was being subtracted from the sum was not actually the oldest value in the array -- the oldest value was, in fact, overwritten by the new value in the first line of the WHILE loop. It was the second-oldest value that was being subtracted from the sum.

EDIT Changed Average and Sum variable to 64-bit floating point to address precision loss over time, on the OP's advice.

Ensuring that the oldest value is subtracted first (once the array is full) gives the expected answer:

PROGRAM
'Program 3 - Simple Moving Average Test
CLEAR
DIM SA(1)
DIM SA0(SMANUM) : REM Moving Average Window as Array
DIM LV1
DIM DV2
LV0 = 0 : REM Counter
DV0 = 0 : REM Average
DV1 = 0 : REM Sum
WHILE(1)
    IF(LV0 >= (SMANUM)) : REM check if we have min num of values
        DV1 = DV1 - SA0(LV0 MOD SMANUM) : REM remove oldest value from sum
    ENDIF
    SA0(LV0 MOD SMANUM) = PLPROBETEMP : REM add Temperature to head of window
    DV1 = DV1 + SA0(LV0 MOD SMANUM) : REM add new value to sum
    IF(LV0 >= (SMANUM)) : REM check if we have min num of values
        DV0 = DV1 / SMANUM : REM calc moving average
        PRINT "Avg: " ; DV0 , " Converted: " ; DV0 * 21.875 - 75
    ENDIF
    LV0 = LV0 + 1 : REM increment counter
WEND

I don't have a running BASIC environment but I tested this in Python and got the same incorrect output for code equivalent to your version and the expected output for code equivalent to the version that I've inserted above.

edited Apr 23 '13 at 22:45

answered Apr 22 '13 at 21:06

Simon

10,679
1
30
44

Thanks so much, this is the solution. You are correct, I was overwriting the oldest value when I obtained the new temperature. I've been banging my head on this for a while, thanks again. – Ryan R Apr 22 '13 at 22:20
Well it looks like I may have jumped the gun. I am still noticing a small offset between the raw and moving average. For example: `Raw: 4.55322 Avg: 4.42033` which is about `2.9C` off. I will do some further investigation tomorrow and comment here with my findings. – Ryan R Apr 22 '13 at 22:42
You could try testing the code as I did by putting a sequence of voltages into an array (say, `V`) and reading each voltage from `V(LV0)`. That way, I could make the values in the first half of `V` equal to (say) `1` and the values in the second half of `V` equal to (say) `5` and observe the moving average that was output based on a consistent stepped sequence of input values. Another way to look at it would be to calculate the sum of the values in the array `SA0` at each time and compare that to `SV1`. That might not be fast enough for production use, but could help spot any remaining bugs. – Simon Apr 22 '13 at 23:09
Thanks Simon, I rewrote the algorithm in C# and confirmed that the indexing of the window is correct in your answer. I left the BASIC version running over night on the controller and noticed the offset grew over time. I suspect there is a loss of precision of the sum variable SV1 which is increasing the offset over time. Since these are 32-bit floating point numbers do you think moving to a 64-bit floating point would solve the issue? – Ryan R Apr 23 '13 at 16:56
It could be a loss-of-precision issue because subtracting `75` in the temperature conversion is losing a little over 8 bits out of the 32. Thus, keeping the sum in degrees C rather than in raw units might reduce the precision loss considerably. Alternatively, I expect that moving to 64-bit floating point would solve the issue. Or, you could periodically recalculate `SV0` from the array every few hundred cycles, which would reset the gradual precision loss. – Simon Apr 23 '13 at 20:16
Thanks Simon, I moved to 64-bit precision on the average and sum variables and that seems to do the trick. I will leave it running over night to confirm but I suspect this is the solution. I will update your answer to reflect this. Thanks again. – Ryan R Apr 23 '13 at 20:20
Hey Simon, if the counter `LV0` overflows and wraps around will this cause the algorithm to break? `LV0` is a 32 bit integer. – Ryan R May 22 '13 at 17:37
Yes, it will, but that is easily fixed. I was initially going to suggest just retaining `LV0 MOD SMANUM` but that would break the test for the minimum number of values. The easiest way to deal with this would be to retain `LV0 MOD (2*SMANUM)` by changing the last line of the `WHILE` loop to `LV0 = (LV0 + 1) MOD (2*SMANUM)`, which should ensure `LV0` never overflows. – Simon May 22 '13 at 20:35
Wouldn't that also break the minimum number of values test 50% of the time? Perhaps a simple `IF` statement after the increment on the last line to reset `LV0` with an offset of `SMANUM` like so: `IF(LV0 > SMANUM) LV0 = SMANUM + (LV0 MOD SMANUM)` – Ryan R May 23 '13 at 15:40
You're right. It would break the minimum number of values test 50% of the time. Your alternative solves the problem. – Simon May 23 '13 at 20:37
Thanks again for your help Simon. Always appreciated. – Ryan R May 23 '13 at 20:39

Simple Moving Average summation/offset issue

1 Answers1

Linked