How to find the first of week from a series of dtype('

Question

I have a series p of timestamps of type: dtype('<M8[ns]')

I am trying to convert it to the first day of the week like so:

p - pd.Timedelta(days=p.dt.dayofweek)

This clearly is not the right answer

TypeError: Invalid type <class 'pandas.core.series.Series'>. Must be int or float.

but what is?

------------Full stack trace -----------------

TypeErrorTraceback (most recent call last)
<ipython-input-699-bb55f007b4d9> in <module>()
      2 p= testDf.PlayDate.drop_duplicates()
      3 #p=pd.to_datetime(p)
----> 4 p - pd.to_timedelta(p.dt.dayofweek, unit='D')
      5 

C:\DS\Installs\Anaconda3\lib\site-packages\pandas\core\ops.py in wrapper(left, right, name, na_op)
    607 
    608         time_converted = _TimeOp.maybe_convert_for_time_op(left, right, name,
--> 609                                                            na_op)
    610 
    611         if time_converted is None:

C:\DS\Installs\Anaconda3\lib\site-packages\pandas\core\ops.py in maybe_convert_for_time_op(cls, left, right, name, na_op)
    567             return None
    568 
--> 569         return cls(left, right, name, na_op)
    570 
    571 

C:\DS\Installs\Anaconda3\lib\site-packages\pandas\core\ops.py in __init__(self, left, right, name, na_op)
    280             left, right = left.align(right, copy=False)
    281 
--> 282         lvalues = self._convert_to_array(left, name=name)
    283         rvalues = self._convert_to_array(right, name=name, other=lvalues)
    284 

C:\DS\Installs\Anaconda3\lib\site-packages\pandas\core\ops.py in _convert_to_array(self, values, name, other)
    396             supplied_dtype = values.dtype
    397         inferred_type = supplied_dtype or lib.infer_dtype(values)
--> 398         if (inferred_type in ('datetime64', 'datetime', 'date', 'time') or
    399                 com.is_datetimetz(inferred_type)):
    400             # if we have a other of timedelta, but use pd.NaT here we

TypeError: data type "datetime" not understood

Can you also add data that reproduces the error? – EFT Jun 27 '17 at 18:12 — EFT, Jun 27 '17 at 18:12

EFT · Accepted Answer · 2017-06-27T22:23:45.953

1

Your current code is trying to make a single timedelta object out of a series, and so the constructor is failing.

p - pd.to_timedelta(p.dt.dayofweek, unit='D')

should do what you want.

Edit:

Ex:

help(pd)
Help on package pandas:

...

VERSION
    0.18.1

...



p = pd.date_range('1/1/2017', '1/23/2017').to_series().reset_index(drop=True)

p.dtype
Out[8]: dtype('<M8[ns]')

print(p.dtype)
datetime64[ns]

p - pd.to_timedelta(p.dt.dayofweek, unit='D')
Out[10]: 
0    2016-12-26
1    2017-01-02
2    2017-01-02
3    2017-01-02
4    2017-01-02
5    2017-01-02
6    2017-01-02
7    2017-01-02
8    2017-01-09
9    2017-01-09
10   2017-01-09
11   2017-01-09
12   2017-01-09
13   2017-01-09
14   2017-01-09
15   2017-01-16
16   2017-01-16
17   2017-01-16
18   2017-01-16
19   2017-01-16
20   2017-01-16
21   2017-01-16
22   2017-01-23
dtype: datetime64[ns]

print(pd.to_timedelta(p.dt.dayofweek, unit='D').dtype)
timedelta64[ns]

Edit2:

To summarize the comment discussion below, the issue still being experienced was due to an out of date version of pandas. With pandas version greater than 0.18.0, the solution above should work.

edited Jun 27 '17 at 22:23

answered Jun 27 '17 at 16:11

EFT

2,359
1
10
11

Thanks this gives the error: `TypeError: data type "datetime" not understood` I did a `print(p.dtype)` and it gave: `datetime64[ns]` as opposed to just typing `p.type` which gives: `dtype(' – Reddspark Jun 27 '17 at 16:42
@user1761806 the difference between `print(p.dtype)` & `p.dtype` shouldn't be relevant; I get the same. It's because the `__str__` and `__repr__` differ: `p.dtype.__str__() Out[241]: 'datetime64[ns]' p.dtype.__repr__() Out[242]: "dtype(' – EFT Jun 27 '17 at 17:00
@user1761806 Could you edit your question to include the full stack trace of the current error? – EFT Jun 27 '17 at 17:13
Done.. I tried to decipher it myself but sadly failed. – Reddspark Jun 27 '17 at 17:36
Ok is this perhaps a python bug? The line that fails is: ` if (inferred_type in ('datetime64', 'datetime', 'date', 'time') or` and the reason why it fails is because `inferred_type = datetime64[ns]` – Reddspark Jun 27 '17 at 21:17
@user1761806 maybe a pandas bug. What version do you have installed currently? It's working on `0.20.2` and `0.18.1` for me, so if your version is earlier, you could try updating, and if that works we might be able to pin down where/if it changes in the patch notes. – EFT Jun 27 '17 at 21:21
I have 0.18. The code above works for you then? (Where p is a datetime64[ns] and you are subtracting a timedelta64[ns])? – Reddspark Jun 27 '17 at 21:27
Thanks - you helped me figure out the issue. Anaconda python packages were out of date. I needed to do a conda update pandas to update them an now it works! If you post it as an answer I'd be more than happy to award you the points. – Reddspark Jun 27 '17 at 22:18
@user1761806 This is my answer that we're commenting on. I've edited in a summary of what we've worked out here though, which should at least make things easier for anyone happening on this later. – EFT Jun 27 '17 at 22:27

How to find the first of week from a series of dtype('

1 Answers1