3

I have a dataframe with an index that also contains the Hour:second:... but instead I just want it to contain the date as year, month, day.

Is it possible for the index to still be a DateTimeIndex but contain only year, month, day?

The current index looks like:

my_index = DatetimeIndex(['2017-08-25', '2017-08-24', '2017-08-23', '2017-08-22',
           '2017-08-21', '2017-08-20', '2017-08-19', '2017-08-18',
           '2017-08-17', '2017-08-16',
           ...
           '2015-07-19', '2015-07-18', '2015-07-17', '2015-07-16',
           '2015-07-15', '2015-07-14', '2015-07-13', '2015-07-12',
           '2015-07-11', '2015-07-10'],
          dtype='datetime64[ns]', length=778, freq=None)

I could do:

only_date_index = [el.date() for el in my_index]

But then if I want to use the resample function in pandas I get the error: TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index'.

Dnaiel
  • 7,622
  • 23
  • 67
  • 126
  • 1
    Try `.normalize()` it will make the times go to midnight, as i believe the `DateTimeIndex` needs the time – Deckerz Aug 29 '17 at 15:47

1 Answers1

8

You need DatetimeIndex.floor:

myIndex = df.index.floor('D')

Sample:

rng = pd.date_range('2017-04-03 15:00:45', periods=10, freq='24T')
df = pd.DataFrame({'a': range(10)}, index=rng)  
print (df)
                     a
2017-04-03 15:00:45  0
2017-04-03 15:24:45  1
2017-04-03 15:48:45  2
2017-04-03 16:12:45  3
2017-04-03 16:36:45  4
2017-04-03 17:00:45  5
2017-04-03 17:24:45  6
2017-04-03 17:48:45  7
2017-04-03 18:12:45  8
2017-04-03 18:36:45  9

myIndex = df.index.floor('D')
print (myIndex)
DatetimeIndex(['2017-04-03', '2017-04-03', '2017-04-03', '2017-04-03',
               '2017-04-03', '2017-04-03', '2017-04-03', '2017-04-03',
               '2017-04-03', '2017-04-03'],
              dtype='datetime64[ns]', freq=None)

Thanks for another solution Deckerz - using DatetimeIndex.normalize:

myIndex = df.index.normalize()
print (myIndex)
DatetimeIndex(['2017-04-03', '2017-04-03', '2017-04-03', '2017-04-03',
               '2017-04-03', '2017-04-03', '2017-04-03', '2017-04-03',
               '2017-04-03', '2017-04-03'],
              dtype='datetime64[ns]', freq=None)

Timings:

ix = pd.date_range('1970-01-01', '2200-01-15', freq='1H')

print (len(ix))
2016481

In [68]: %timeit (ix.normalize())
10 loops, best of 3: 178 ms per loop

In [69]: %timeit (ix.floor('d'))
10 loops, best of 3: 38.4 ms per loop

#solution from Dror (https://stackoverflow.com/questions/45954497/in-pandas-group-by-date-from-datetimeindex)
In [70]: %timeit pd.to_datetime(ix.date)
1 loop, best of 3: 5.09 s per loop
jezrael
  • 822,522
  • 95
  • 1,334
  • 1,252
  • `.normalize()` also does this and is an embedded function for DateTimeIndex arrays but not single instances – Deckerz Aug 29 '17 at 15:49