I'm a programmer who is interested in processing and analyzing time-series data. I know basic statistics and math, but I'm afraid that's all.
Can you please recommend good books and/or articles that does not require Ph.D. to understand them?
As for my concrete tasks - I want to be able to spot trends, eliminate outliers, be able to make predictions and calculate stats over a range of values. We have quite a bit of events coming off our systems.
I started reading "Introduction to Time Series and Forecasting" by Brockwell and Davis - and I'm completely lost in math.
update on outliers by outliers I mean data points that doesn't necessarily make sense. e.g. the exchange rate is 1.5$(+-10 cents) for a pound on average, but a guy around the corner offers 1.09$ and says he's completely legit.
- 8,328
- 17
- 68
- 113
-
can you explain votes to close? may be there's a question I haven't found? – chester89 Sep 27 '16 at 20:32
2 Answers
I've found the NIST Engineering Statistics Handbook's chapter on time series to be a simple and clear introduction to basic time series modeling. It discusses exponential smoothing, auto-regressive, moving average, and eventually ARMA time series modeling. These can be used for trend analysis and possibly prediction, subject to validation.
Outlier/anomaly detection is a much different task; the NIST book doesn't have much on this. It would be helpful to know what kind of outliers you are trying to detect.
- 86
- 2
I've gone through numerous books and articles and here are my findings. May be they will help others like me.
Regarding theory - I found an article An Introductory Study on Time Series Modeling and Forecasting very well written. That doesn't mean I understood all of its contents, but it's a really good overview of available time series models.
If you're like me and like to see some actual code - there's article series on QuantStart. Examples are in R, but I guess many of them are portable to Python.
I can highly recommend QuantStart blog by Michael Halls-Moore, I found articles easy to read and the author has done a great job trying not to overwhelm a reader with math. I also read Michael's first book and it's a good one for a beginner in the space like me.
Textbooks on the topic are extremely hard for me to read. I tried Time Series Analysis by Hamilton, but haven't gotten far.
Regarding outlier detection I mentioned - I've found this question on SO and its stats counterpart. By the looks of it, it's not something you can study and implement in a couple of evenings, at least not for me.
-
I know there's been 2 years since my question, but it looks to me that detecting outliers is a very common task. Here's a decent article with examples in Python https://towardsdatascience.com/5-ways-to-detect-outliers-that-every-data-scientist-should-know-python-code-70a54335a623 – chester89 Mar 27 '19 at 09:55