1

Here is discussion about correct interpretation of NuPIC output which I would like to extend. First I will provide short summary and then ask another question.

Consider following output:

step,original,prediction,anomaly score 
175,0,0.0,0.32500000000000001
176,62,52.0,0.65000000000000002
177,402,0.0,1.0
178,0,0.0,0.125
179,402,0.0,1.0
180,0,0.0,0.0
181,3,402.0,0.050000000000000003
182,50,52.0,0.10000000000000001
183,68,13.0,0.90000000000000002 

This is output of one step ahead prediction without using inference shifter. It basically mean that the prediction made at step N is for step N+1. Or in another words if the prediction is perfectly right then prediction value at step N should correspond to the original value at step N+1.

Anomaly score can be viewed as confidence of prediction. For example, NuPIC might only be 23% confident in the best prediction it gives, in which case the anomaly score could be very high. This is the case of step 179 where prediction is 0 and the original value on step 180 is 0. Note that anomaly score on step 179 is 1.0. It means that NuPIC was not confident in prediction, despite that the prediction was correct.

Opposite situation happens on step 180 where prediction is 0 and the original value on step 181 is 3. Note that anomaly score on step 180 is 0. That means that NuPIC was quiet confident in prediction but it was not correct.

Questions:

  1. Does anomaly score on given step also counts with the original value on given step? For example anomaly score on this step 181,3,402.0,0.050000000000000003 take into account that 3 is the original value? Or it is computed without respect to this value?

  2. Is it possible to compute some kind of debug information reading prediction and anomaly score? I mean something like this from NuPIC perspective: I'm 23% sure that next value will be 10, I'm 27% sure that next value will be 20, I'm 50% sure that next value will be 30.

  3. Is OK to predict data for zero steps forward if I'm just interested in the prediction accuracy?

  4. Does NuPIC make some kind of look back? I mean if NuPIC was at step 180 confident that next value will be 0 but later it shows that it was mistake does NuPIC somehow recount the anomaly score from step 180 for further data processing? Or this is done automatically in HTM?

Wakan Tanka
  • 7,542
  • 16
  • 69
  • 122

0 Answers0