Why can't I access date time data within pandas?

Question

df.reset_index()

DateTime    Demand  Grid1 Load  Grid1 Loss  Grid1 Loss Prophet Daily    
0   01/12/2019 00:00    314.396236  407.67600   21.55210    -3.938760   25.64150    23.9576 -2.50138    
1   01/12/2019 01:00    305.691739  397.96600   20.71190    -4.419430   25.19200    23.9586 -2.48713    
2   01/12/2019 02:00    300.431535  392.90400   20.27340    -4.492290   25.15940    23.9595 -2.46375    


df['DateTime'] = pd.to_datetime(df['DateTime'], format = '%d/%m/%Y/n%H:%H')

I am trying to work with the data in the DateTime column but every time I try to access it in any way it returns the following error:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~\Anaconda3\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
   2656             try:
-> 2657                 return self._engine.get_loc(key)
   2658             except KeyError:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'DateTime'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-28-80bbba28691a> in <module>
----> 1 df['DateTime'] = pd.to_datetime(df['DateTime'], format = '%d/%m/%Y/n%H:%H')

~\Anaconda3\lib\site-packages\pandas\core\frame.py in __getitem__(self, key)
   2925             if self.columns.nlevels > 1:
   2926                 return self._getitem_multilevel(key)
-> 2927             indexer = self.columns.get_loc(key)
   2928             if is_integer(indexer):
   2929                 indexer = [indexer]

~\Anaconda3\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
   2657                 return self._engine.get_loc(key)
   2658             except KeyError:
-> 2659                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   2660         indexer = self.get_indexer([key], method=method, tolerance=tolerance)
   2661         if indexer.ndim > 1 or indexer.size > 1:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'DateTime'

No matter what I try I get the same error, even to try find the data type or to access the column data using

df['DateTime']
type(df[DateTime'])

Any ideas on what I should try to fix this?

You need to persist your `reset_index`. See my answer below. — Prayson W. Daniel, Feb 27 '21 at 09:40

score 1 · Answer 1 · answered Feb 27 '21 at 09:30

1

You are getting error because you didn't give inplace parameter in reset_index() method

So due to this 'DateTime' is consider as index instead of column in your dataframe

So Write:-

df.reset_index(inplace=True)

answered Feb 27 '21 at 09:30

Anurag Dabas

23,866
9
21
41

`inplace=True` will work but it can be harmful https://stackoverflow.com/questions/45570984/in-pandas-is-inplace-true-considered-harmful-or-not – Prayson W. Daniel Feb 27 '21 at 09:43

score 1 · Answer 2 · answered Feb 27 '21 at 09:37

The reason is that your reset_index is temporarily. You need to persist the changes by assigning your reset to your DataFrame.

...
df = df.reset_index()

I would not recommend using inplace=True as it's bad practice. Reasons to show that inplace is bad see another SO forum "is in place hamrful"?

score 0 · Answer 3 · answered Feb 27 '21 at 10:23

0

Without resetting index:

df.index = pd.to_datetime(df.index, format='%d/%m/%Y %H:%M')

answered Feb 27 '21 at 10:23

Razor1ty

51
6

Why can't I access date time data within pandas?

3 Answers3