By the api I might get batches of sensor data with requests(say about 200ms per batch and sampling rate at 400Hz).
With a butter lowpass I have got the way to count what I want on static demo dataset like below.
But in real world the data from the api comes continuously. I need to record all of them,detect and count the peakspeaks+=1,print(peaks) at the same time in Jupyter notebook. I tryed open two notebooks,one for record in sqlite3 :memory:, the another read last 5 seconds data and counts. But it seems that sqlite3 can not share memory db between two notebooks(maybe different threads), and IO with db on disk is really costly. I think the best way might be using something like FiFO quene and threading for two job at the sametime in memory. Is there any framework for these real time data analysis jobs on api or serials when data comes continuously and You must consider the time series features in moving window data.iloc[-window_length:,:]?
import pandas as pd
import seaborn as sns
from scipy import signal
## load static data
df=pd.read_excel('g 2022-09-19_17-18-56.xls')
sample_rate=df.index[-1]/df.iloc[-1,0]
print(sample_rate,'Hz')
## plot sensor_data
ax=sns.lineplot(data=df.iloc[:,:],x='Time (s)',y='Absolute acceleration (m/s^2)')
ax.figure.set_size_inches(20,10)
## butter lowpass
b, a = signal.butter(6, 0.005, 'lowpass')
filtedData = signal.filtfilt(b, a, df.loc[:,'Absolute acceleration (m/s^2)']) #data为要过滤的信号
ax=sns.lineplot(x=df['Time (s)'],y=filtedData)
ax.figure.set_size_inches(20,10)
## find peaks
peaks,_=signal.find_peaks(x=-filtedData,distance=400)
ax=sns.scatterplot(x=df['Time (s)'][peaks],y=filtedData[peaks],color='purple',marker='s',)
ax.figure.set_size_inches(20,10)
peaks_count=len(peaks)
print(f"There are {peaks_count} peaks in static data.")
#steaming_data_=requests.get()
#collecting all the data and detect
#while 1:
# if peaks detected:
# print(peaks count)
