Here is discussion about correct interpretation of NuPIC output which I would like to extend. First I will provide short summary and then ask another question.
Consider following output:
step,original,prediction,anomaly score
175,0,0.0,0.32500000000000001
176,62,52.0,0.65000000000000002
177,402,0.0,1.0
178,0,0.0,0.125
179,402,0.0,1.0
180,0,0.0,0.0
181,3,402.0,0.050000000000000003
182,50,52.0,0.10000000000000001
183,68,13.0,0.90000000000000002
This is output of one step ahead prediction without using inference shifter. It basically mean that the prediction made at step N
is for step N+1
. Or in another words if the prediction is perfectly right then prediction value at step N
should correspond to the original value at step N+1
.
Anomaly score can be viewed as confidence of prediction. For example, NuPIC might only be 23% confident in the best prediction it gives, in which case the anomaly score could be very high. This is the case of step 179
where prediction is 0 and the original value on step 180
is 0
. Note that anomaly score on step 179
is 1.0
. It means that NuPIC was not confident in prediction, despite that the prediction was correct.
Opposite situation happens on step 180
where prediction is 0
and the original value on step 181
is 3
. Note that anomaly score on step 180
is 0
. That means that NuPIC was quiet confident in prediction but it was not correct.
Questions:
Does anomaly score on given step also counts with the original value on given step? For example anomaly score on this step
181,3,402.0,0.050000000000000003
take into account that3
is the original value? Or it is computed without respect to this value?Is it possible to compute some kind of debug information reading prediction and anomaly score? I mean something like this from NuPIC perspective: I'm 23% sure that next value will be 10, I'm 27% sure that next value will be 20, I'm 50% sure that next value will be 30.
Is OK to predict data for zero steps forward if I'm just interested in the prediction accuracy?
Does NuPIC make some kind of look back? I mean if NuPIC was at step
180
confident that next value will be0
but later it shows that it was mistake does NuPIC somehow recount the anomaly score from step180
for further data processing? Or this is done automatically in HTM?