0

I am trying to create a table structure through tb.IsDescription class, then create a .h5 file and populate it with a Pandas Dataframe with Datetime index, using TsTables package. I have already tested the Dataframe and the date time Indexing and both seem to be fine. I believe the issue is with the TsTable package, as it remains 'Unused import statement'. The error I get is: " AttributeError: module 'pandas.tseries' has no attribute 'index' ". The reason I am using the TsTAble is that I have heard it is faster than other modules. Any suggestions how to resolve this issue, or any substitute method?

import numpy as np
import pandas as pd
import tables as tb                  
import datetime as dt

path = r'C:\Users\--------\PycharmProjects\pythonProject2'



no = 5000000                       # number of time steps
co = 3                               # number of time series
interval = 1. / (12 * 30 * 24 * 60)    # the time interval as a year fraction   
vol = 0.2                                 # volatility

rn = np.random.standard_normal((no, co))
rn[0] = 0.0                                 # sets the initial random numbers to zero

paths = 100 * np.exp(np.cumsum(-0.5 * vol ** 2 * interval + vol * np.sqrt(interval) * rn, axis=0))
                                           # simulation based on an Euler discretization

paths[0] = 100                              # Sets the initial values of the paths to 100

dr = pd.date_range('2019-1-1', periods=no, freq='1s')
print(dr[-6:])                            # the date range appears fine

df = pd.DataFrame(paths, index=dr, columns=['ts1', 'ts2', 'ts3'])
print(df.info(verbose=True))             # df is pandas Dataframe and appears fine
print(df.head())                       # tested a fraction of the data, it is fine


import tstables as tstab             # I get Unused import statement

class ts_desc(tb.IsDescription):       
    timestamp = tb.Int64Col(pos=0)         # The column for the timestamps
    ts1 = tb.Float64Col(pos=1)                 # The column to store numerical data
    ts2 = tb.Float64Col(pos=2)
    ts3 = tb.Float64Col(pos=3)

h5 = tb.open_file(path + 'tstab.h5', 'w')

ts = h5.create_ts('/', 'ts', ts_desc)
          
ts.append(df)                         # !!!!! the error I get is from this code line !!!!
               # value error raised is: if rows.index.__class__ != pandas.tseries.index.DatetimeIndex:
                 AttributeError: module 'pandas.tseries' has no attribute 'index'      `
kcw78
  • 7,131
  • 3
  • 12
  • 44
Rezhes
  • 1
  • 1

1 Answers1

0

I suspect you have run into a version compatibility issue between tstables and your pandas versions (assuming you are running any recent pandas version). Based on the tstables PyPI page, the last release of tstables was in 2015. A check of the tstables github project page shows there was an issue with Pandas 0.20.3 and use of datetime. The error message is the same as yours: module 'pandas.tseries' has no attribute 'index' in tstables See this: tstables breaks down with Pandas 20.3

The issue has a link to another build that works with Pandas 0.20.3. Development notes state "Removed 'convert_datetime64' parameter on line 245". Not sure if it will work with more recent versions, but worth a try. See this: schwed2 tstables build

If that doesn't solve the problem, I suggest running the simple examples provided or run the setup tests. (Note: I could not find the bpi_2014_01.csv file to test the bitcoin/bpi example.)

Good luck.

kcw78
  • 7,131
  • 3
  • 12
  • 44
  • Thank u sir @kcw78, besides the removal mentioned in the 'schwed2 tstables build' above, I had to comment out two lines below regarding "ValueError" in lines 239 and 240, and this solved the issue: ' def append(self,rows,convert_strings=False): ... ... #if rows.index.__class__ != pandas.tseries.index.DatetimeIndex: #raise ValueError('when rows is a DataFrame, the index must be a DatetimeIndex.') # Convert to records records = rows.to_records(index=True) #, convert_datetime64=False) ' – Rezhes Nov 16 '22 at 12:41