2

I have a dataset containing around 1000 different time series. Some of these are showing clear periodicity, and some are not.

I want to be able to automatically determine if a time series has clear periodicity in it, so I know if I need to do seasonal decomposition of it before applying some outlier methods.

Here is a signal with daily periodicity, each sample is taken with 15 minute interval.

enter image description here

In order for me to try and automatically determine if there is daily periodicity, I have tried using different methods. The first approach is using detection function in library from kats.

 from kats.consts import TimeSeriesData
 from kats.detectors.seasonality import FFTDetector, ACFDetector
 def detect_seasonality(df,feature,time_col,detector_type):

    df_kpi = df[[feature]].reset_index().rename(columns={feature:'value'})
    ts = TimeSeriesData(df_kpi,time_col_name=time_col)

    if detector_type == 'fft':
        detector = FFTDetector(ts)
    elif detector_type == 'acf':
        detector = ACFDetector(ts)
    else:
        raise Exception("Detector types are fft or acf")

    detection = detector.detector()
    seasonality_presence = detection['seasonality_presence']
    return seasonality_presence

This approach returned "False" seasonality presence, both using fft and acf detector.

Another approach is using fft

import numpy as np
import scipy.signal

from matplotlib import pyplot as plt

L = np.array(df[kpi_of_interest].values)
L -= np.mean(L)
# Window signal
L *= scipy.signal.windows.hann(len(L))

fft = np.fft.rfft(L, norm="ortho")

plt.figure()
plt.plot(abs(fft))

enter image description here

But here we don't see any clear way to determine the daily periodicity I expected.

So in order for me to automatically detect the daily periodicity, are there any other better methods to apply here? Are there any necessary preprocessing steps needed for me in beforehand? Or could it simply be a lack of data? I only have around 10 days of data for each time series.

mkrieger1
  • 19,194
  • 5
  • 54
  • 65
kspr
  • 980
  • 9
  • 23
  • It looks similar to [this post](https://stackoverflow.com/questions/68050756/is-there-an-easy-way-of-finding-frequency-of-envelopes-in-sound-signals/68335518#68335518). Note that the result of "normalised" rfft frequency is relative to the signal rate which is not provided in your case so this is likely why the result seems not useful. I guess the first biggest pic in the left should be primary frequency and the other are probably harmonics. – Jérôme Richard Sep 08 '21 at 14:08
  • Hello, you may try to convolve the signal with itself https://stackoverflow.com/questions/49531952/find-period-of-a-signal-out-of-the-fft/49545394#49545394 or compute the energy or power spectrum and select the peaks https://stackoverflow.com/questions/54714169/why-are-frequency-values-rounded-in-signal-using-fft/54775867#54775867 . Yet the signal you provided looks noisy. A low-pass filter could be applied first to remove a high frequency noise. – francis Sep 09 '21 at 17:53

0 Answers0