1

I know a bit about the different types of datetime objects in numpy but I don't fully understand it.

I want to use pandas.DataFrame.update() to update a timestamp column with some values of timestamps from another dataframe. The dtype of the "other" timestamps is <M8[ns]. When I update the original dataframe with that values the result are not timestamps but just big integers like 1444435200000000000.

The MWE does not reproduce the error because I don't know how to generate <M8[ns] values.

So the main question is how can I deal with that problem to update a timestamp column with timestamps of "different" types?

The secondary question would be how such <M8[ns] timestamps are generated? Maybe I can find the place in my code where they are created. Currently I just do pandas.to_datetime(arg=df.time_strings, format='%Y-%m-%d').

#!/usr/bin/env python3
import pandas as pd

# !!! The MWE does **not** reproduce the error because
# !!! I don't know how to generate `<M8[ns]` values.

df = pd.DataFrame({
    'ID': [1, 2],
    'A': [
        pd.Timestamp('2022-09-11 19:04:47'),
        pd.Timestamp('2022-09-11 11:57:39'),
    ]})
print(df.A.dtype)  # dtype('O')


x = pd.DataFrame({
    'ID': [2],
    'A': [
        pd.Timestamp('2016-09-11 00:01:22'),  # this should be of type "<M8[ns]"
    ]})

print(x.A.dtype)  # dtype('<M8[ns]')

df.update(x)
print(df)
FObersteiner
  • 22,500
  • 8
  • 42
  • 72
buhtz
  • 10,774
  • 18
  • 76
  • 149
  • 1
    which pandas version did you use? This works fine on 1.4.4 – mozway Sep 13 '22 at 12:01
  • The MWE does **not** reproduce the error because I don't know how to generate ` – buhtz Sep 13 '22 at 12:05
  • 1
    How did you generate the issue in the first place? – mozway Sep 13 '22 at 12:08
  • Can't answer that here. To complexe in productive code. I wasn't able to reproduce (generate – buhtz Sep 13 '22 at 12:21
  • 1
    Date/time is represented as nanosecond since the Unix epoch by the datetime64[ns] types. For instance 1444435200000000000 is Oct 10 2015 00:00 UTC - so if you end up with integers, you can just convert back to datetime. – FObersteiner Sep 13 '22 at 15:49
  • btw. could this be a matter of how things are *shown* in your output? - try `print(repr(x.A.dtype), str(x.A.dtype))` to see what I mean. Also noted in [this answer](https://stackoverflow.com/a/67242219/10197418) to the Q&A you linked. – FObersteiner Sep 14 '22 at 13:05

0 Answers0