0

I recently posed a question on how to implement an algorithm which adaptively changes a threshold in real time, so that a time series reaches that threshold every N seconds. I was told that if my time series has a constant time interval (it does), I could take the absolute value, reverse sort it, and find the index in the array that gives me the average time resolution I want. I implemented it in MATLAB in this manner:

x = abs(timeseries); % Get the absolute value
x = flipud(sort(x)); % Reverse sort
N = length(x); % Size of the time series
idx = round(N/goal_time); % Find the right index
threshold = x(idx); % Set the threshold

Where goal time is the average time I want to detect a 'hit' (timeseries > threshold). My problem is two fold. I can't tell if the algorithm is not accurate enough, or if my data is too noisy (when using it, the average time I get hits doesn't really approximate my goal very well). And secondly, how would I modify this algorith to calculate a 'hit' time, where a hit is defined as the time series dipping below a threshold?

user1566200
  • 1,826
  • 4
  • 27
  • 47
  • Some questions. 1) What is a Hit? Does [9,11,11,9] count one hit or two? 2) If the algorithm is "inaccurate", could you explain the inaccuracity maybe giving some example data? I don't see any cause for an inaccuracity. Probably the algorithm does not strictly implement what you want? – Daniel Jul 18 '14 at 18:38
  • The `quantile` function might be usefull, statistics toolbox available? – Daniel Jul 18 '14 at 18:39
  • `when using it, the average time I get hits doesn't really approximate my goal very well` Define your goal please? I can see [your last question is here](http://stackoverflow.com/questions/24725578/adjustment-algorithm). Why do you think there should be a hit every one minute? If it's hard to describe your data, may be you can at least post a **graph** of it? – Apiwat Chantawibul Jul 18 '14 at 18:59
  • Sorry for being vague. I am recording the time series as it comes in every 3 seconds. If an incoming number is greater than my threshold, that would register as hit. – user1566200 Jul 18 '14 at 19:16
  • Accidentally hit 'enter' instead of finishing my comment. In my actual data, it's extremely noisy, so let's say I want my goal time to be every 10 seconds on average I want a hit. If I run my time series through this algorithm, I might get an average time of 5 seconds. Set my goal at 15 seconds. Get an average time of 20 seconds. Just varies a bunch, and I'm not sure if it's the algorithm, or the noisiness of my data. Also unsure of how to use this algorithm with a negative threshold. – user1566200 Jul 18 '14 at 19:18
  • Have you tried removing the noise? whether using [Fourier transform](http://wiki.audacityteam.org/wiki/How_Noise_Removal_Works) or [not using](http://stackoverflow.com/questions/6873683/matlab-signal-noise-removal). I think if you have a suspected frequency you are interested in, the Fourier transform will be very useful. – Apiwat Chantawibul Jul 18 '14 at 20:41
  • In image processing, to adaptively change the threshold, there is something called as "Otsu's method". Read that, may be you will get some ideas. – Autonomous Jul 19 '14 at 01:26
  • Alright, thanks for the replies. I think it is indeed the noise in my data, not the algorithm. Also, my time interval is 3 seconds, so the code should actually read: `round(N/(goal_time/3))` – user1566200 Jul 19 '14 at 17:26
  • I'm still having trouble figuring out how to use a similar method to find a negative threshold (a 'hit' when the value drops *below* the threshold). Any suggestions? – user1566200 Jul 19 '14 at 17:26
  • Nevermind. Figured it out. – user1566200 Jul 19 '14 at 20:36

0 Answers0