0

I am new to Pandas. I see it is useful for time series data but only when the sampled interval is a standard unit (eg., minutes, hours, days). What if I have data sampled at much faster but unusual rates such as data acquisition? eg. 512 Hz (sample interval ~1.95 ms)

I can get by without Pandas but just wondered if it is suitable for this type of time series where the sample interval is not an integral number of standard time units.

EDIT: My apologies for being imprecise. I managed to get Pandas to understand engineering-like time series (where we are more used to specifying sampling frequency in Hz than long periods such as days, weeks, months etc) by the following code ...

import pandas as pd

fs = 256. #Hz
period = str(int(round(1000000./fs)))+'U'    #microseconds
rng = pd.date_range('1/1/2011', periods=len(dat), freq=period)
serialData = pd.Series(data,index = rng)

Assuming the record started zero hour 1/1/2011 this worked and got the correct sampling frequency.

However, it is not at all obvious from the documentation how to do this.

I also ran into problems when trying to resample the data at 512Hz (double the frequency). I got this error message:

C:\Users\Andrew\Anaconda\lib\site-packages\pandas\core\format.py:1872: RuntimeWarning: invalid value encountered in greater
has_large_values = (abs_vals > 1e8).any() C:\Users\Andrew\Anaconda\lib\site-packages\pandas\core\format.py:1873: RuntimeWarning: invalid value encountered in less has_small_values = ((abs_vals < 10 ** (-self.digits)) & C:\Users\Andrew\Anaconda\lib\site-packages\pandas\core\format.py:1874: RuntimeWarning: invalid value encountered in greater (abs_vals > 0)).any()

If I change the sample period from '3906U' to '2S' and resample to '1S' (doubling the sampling frequency) I do not get this error.

My data is 8 million samples long. If I try and plot the Pandas TimeSeries object I get a memory error.

I do not get these problems if I just treat the data as a 1D numpy array. So I think I will stick with numpy for the engineering time-series I normally work with.

PS. My edit was just to try and help anyone trying to do something similar with long data and Hz/KHz/MHz etc. based sampling.

ngrashia
  • 9,869
  • 5
  • 43
  • 58
Andrew Walsh
  • 29
  • 3
  • Of course it's suitable. The examples you're referring to are simply shown there, because they are conceptually much easier to understand for anyone, not just for people who know what a Hz is. Also, welcome to StackOverflow. You might want to consider reading the first 4 bulleted items on [What topics can I ask about here?](https://stackoverflow.com/help/on-topic) to get better help in the future. – Oliver W. Mar 07 '15 at 12:52
  • I'm voting to close this question as off-topic because it doesn't cover any of the 4 items explained in the ["What topics can I ask about here?"](https://stackoverflow.com/help/on-topic) page: it's not about (a) a specific programming problem, or (b) a software algorithm, or (c) software tools commonly used by programmers; (d) a practical, answerable problem that is unique to software development. – Oliver W. Mar 07 '15 at 12:57
  • pandas could handle FFT(and way more stuffs). Refer to [10.1. Analyzing the frequency components of a signal with a Fast Fourier Transform][1]. [1]: http://stackoverflow.com/questions/25735153/plotting-a-fast-fourier-transform-in-python – su79eu7k Mar 07 '15 at 14:56

0 Answers0