1

I am not sure if this is a bug or if it's by design-- perhaps I am missing something and the ohlc aggregator isn't supposed to work with dataframes. Perhaps this behavior is by design because a dataframe with anything other than an index column and a price column could yield strange results? Other aggregators (mean,stdev, etc.) work with a dataframe. In any case, I'm trying to get OHLC from this data, and converting to a timeseries doesn't seem to work either.

Here's an example:

import pandas as pd
rng = pd.date_range('1/1/2012', periods=1000, freq='S')

ts = pd.Series(randint(0, 500, len(rng)), index=rng)
df = pd.DataFrame(randint(0,500, len(rng)), index=rng)

ts.resample('5Min', how='ohlc') # works great
df.resample('5Min', how='ohlc') # throws a "NotImplementedError"

newts = pd.TimeSeries(df) #am I missing an index command in this line?
# the above line yields this error "TypeError: Only valid with DatetimeIndex or
  PeriodIndex"

Full NotImplementedError paste:

NotImplementedError                       Traceback (most recent call last)
/home/jeff/<ipython-input-7-85a274cc0d8c> in <module>()
----> 1 df.resample('5Min', how='ohlc')

/usr/local/lib/python2.7/dist-packages/pandas-0.9.2.dev-py2.7-linux-x86_64.egg/pandas/core/generic.pyc in resample(self, rule, how, axis, fill_method, closed, label, convention, kind, loffset, limit, base)
    231                               fill_method=fill_method, convention=convention,
    232                               limit=limit, base=base)
--> 233         return sampler.resample(self)
    234 
    235     def first(self, offset):

/usr/local/lib/python2.7/dist-packages/pandas-0.9.2.dev-py2.7-linux-x86_64.egg/pandas/tseries/resample.pyc in resample(self, obj)
     66 
     67         if isinstance(axis, DatetimeIndex):
---> 68             rs = self._resample_timestamps(obj)
     69         elif isinstance(axis, PeriodIndex):
     70             offset = to_offset(self.freq)

/usr/local/lib/python2.7/dist-packages/pandas-0.9.2.dev-py2.7-linux-x86_64.egg/pandas/tseries/resample.pyc in _resample_timestamps(self, obj)
    189             if len(grouper.binlabels) < len(axlabels) or self.how is not None:
    190                 grouped = obj.groupby(grouper, axis=self.axis)
--> 191                 result = grouped.aggregate(self._agg_method)
    192             else:
    193                 # upsampling shortcut


/usr/local/lib/python2.7/dist-packages/pandas-0.9.2.dev-py2.7-linux-x86_64.egg/pandas/core/groupby.pyc in aggregate(self, arg, *args, **kwargs)
   1538         """
   1539         if isinstance(arg, basestring):
-> 1540             return getattr(self, arg)(*args, **kwargs)
   1541 
   1542         result = {}

/usr/local/lib/python2.7/dist-packages/pandas-0.9.2.dev-py2.7-linux-x86_64.egg/pandas/core/groupby.pyc in ohlc(self)
    384         For multiple groupings, the result index will be a MultiIndex
    385         """
--> 386         return self._cython_agg_general('ohlc')
    387 
    388     def nth(self, n):

/usr/local/lib/python2.7/dist-packages/pandas-0.9.2.dev-py2.7-linux-x86_64.egg/pandas/core/groupby.pyc in _cython_agg_general(self, how, numeric_only)
   1452 
   1453     def _cython_agg_general(self, how, numeric_only=True):
-> 1454         new_blocks = self._cython_agg_blocks(how, numeric_only=numeric_only)
   1455         return self._wrap_agged_blocks(new_blocks)
   1456 

/usr/local/lib/python2.7/dist-packages/pandas-0.9.2.dev-py2.7-linux-x86_64.egg/pandas/core/groupby.pyc in _cython_agg_blocks(self, how, numeric_only)
   1490                 values = com.ensure_float(values)
   1491 
-> 1492             result, _ = self.grouper.aggregate(values, how, axis=agg_axis)
   1493             newb = make_block(result, block.items, block.ref_items)
   1494             new_blocks.append(newb)

/usr/local/lib/python2.7/dist-packages/pandas-0.9.2.dev-py2.7-linux-x86_64.egg/pandas/core/groupby.pyc in aggregate(self, values, how, axis)
    730                 values = values.swapaxes(0, axis)
    731             if arity > 1:
--> 732                 raise NotImplementedError
    733             out_shape = (self.ngroups,) + values.shape[1:]
    734 

NotImplementedError: 
Jeff
  • 377
  • 4
  • 14
  • Sounds like it's not (yet) been implemented... – Andy Hayden Nov 20 '12 at 00:01
  • That may be the case, Hayden. If that's true, I guess I have to figured out how to properly convert my dataframe into a timeseries that I can resample. So far I have not been successful at that either. – Jeff Nov 20 '12 at 00:12
  • 1
    I am able to get the desired result by converting my dataframe to a timeseries using this command: "ts = pd.TimeSeries(df[0])" , and then I can resample the timeseries. Not as elegant as doing it straight from the dataframe, but it works for now. – Jeff Nov 20 '12 at 00:51
  • I had been planning to make it yield hierarchical columns, but looks like I haven't got around to it yet. http://github.com/pydata/pandas/issues/2320 – Wes McKinney Nov 21 '12 at 22:06

1 Answers1

3

You can resample over an individual column (since each of these is a timeseries):

In [9]: df[0].resample('5Min', how='ohlc')
Out[9]: 
                     open  high  low  close
2012-01-01 00:00:00   136   136  136    136
2012-01-01 00:05:00   462   499    0    451
2012-01-01 00:10:00   209   499    0    495
2012-01-01 00:15:00    25   499    0    344
2012-01-01 00:20:00   200   498    0    199


In [10]: type(df[0])
Out[10]: pandas.core.series.TimeSeries

It's not clear to me how this should output for a larger DataFrames (with multiple columns), but perhaps you could make a Panel:

In [11]: newts = Panel(dict((col, df[col].resample('5Min', how='ohlc'))
                                for col in df.columns))

In [12]: newts[0]
Out[12]: 
                     open  high  low  close
2012-01-01 00:00:00   136   136  136    136
2012-01-01 00:05:00   462   499    0    451
2012-01-01 00:10:00   209   499    0    495
2012-01-01 00:15:00    25   499    0    344
2012-01-01 00:20:00   200   498    0    199

Note: Perhaps there is a canonical output for resampling a DataFrame and it is yet to be implemented?

Andy Hayden
  • 359,921
  • 101
  • 625
  • 535
  • Resampling over an individual column was what I was looking for. Works perfectly. Thank you, Hayden. – Jeff Nov 21 '12 at 02:15