1

I'm looking for a neat way to detect particular events in time series data.

In my case, an event might consist of a value changing by more than a certain amount from one sample to the next, or it might consist of a sample being (for example) greater than a threshold while another parameter is less than another threshold.

e.g. imagine a time series list in which I've got three parameters; a timestamp, some temperature data and some humidity data:

time_series = []
#                  time, temp, humidity
time_series.append([0.0, 12.5, 87.5])
time_series.append([0.1, 12.8, 92.5])
time_series.append([0.2, 12.9, 95.5])

Obviously a useful time series would be much longer than this.

I can obviously loop through this data checking each row (and potentially the previous row) to see if it meets my criteria, but I'm wondering if there's a neat library or technique that I can use to search time series data for particular events - especially where an event might be defined as a function of a number of contiguous samples, or a function of samples in more than one column.

Does anyone know of such a library or technique?

Trenton McKinney
  • 56,955
  • 33
  • 144
  • 158
Jon Mills
  • 1,855
  • 4
  • 19
  • 28
  • Whenever you're about to add a record, check the past N rows and emit the necessary events? That would work too, not sure if a library already exists. – Simeon Visser Dec 03 '14 at 17:22
  • Thanks Simeon. That's my current solution, but it always feels like a bit of a cludge. I know there are quite a few Python libraries that can handle tabular data (NumPy, SimPy, Pandas), but I was wondering if any of them (or something else) can handle this sort of requirement in a more generic manner. – Jon Mills Dec 04 '14 at 08:13

1 Answers1

0

You might like to investigate pandas, which includes time series tools see this pandas doc.

I think that what you are trying to do is take "slices" through the data. [This link on earthpy.org] (http://earthpy.org/pandas-basics.html) has a nice introduction to using time series data with pandas, and if you follow down through the examples it shows how to take out slices, which I think would correspond to pulling out parameters that exceed thresholds, etc. in your data.

Breaks Software
  • 1,721
  • 1
  • 10
  • 15
  • Please give a more complete answer, explain the link and how it might be used. – damian Dec 03 '14 at 18:07
  • Thanks @kingdamian42. I've updated my answer with further comments. I've avoided being too detailed in part because the original post is a rather generic request, and in part because I am only an occasional user of pandas myself. Hopefully the linked pages will be a useful guide that can prompt more specific requests. – Breaks Software Dec 04 '14 at 13:13
  • Thanks for the link to the more specific Pandas doc. That looks really useful - or certainly close to something I need. It seems like Pandas may be the tool I need. – Jon Mills Dec 06 '14 at 08:18