1

I'm new here and need help in understanding how i can work with timestamps to datetime objects that are used in pandas. I saved some data using websockets in a csv file and loaded that csv file into a pandas dataframe. In my timestamp column i'm getting contents like [2018-02-04T07:49:36.867Z, 2018-02-04T07:49:56.931Z and so on].

I have to manipulate the other data columns using the time data, like re-sampling (using pandas) over certain durations say 1 min, 3 min etc. But I can't apply re-sampling as the date and time is not in correct format, like this [20180204 07:49:56.931, 20180204 07:49:56:931 and so on].

How to achieve this transformation in pandas/python. Is it just just simple first string manipulation that i just remove these unwanted characters and then apply the datetime transformation. Any help on how to proceed would be helpful.

I don't even know where to start as I have never come across this type of format.

Flika205
  • 532
  • 1
  • 3
  • 18
Mr. Confused
  • 245
  • 2
  • 11
  • There are several ways you can do this. For example Regular Expressions in python, or simple string replace. But I would suggest something like this: https://stackoverflow.com/questions/466345/converting-string-into-datetime and then use the string format methods of the datetime object. Docs here: https://docs.python.org/2/library/datetime.html#module-datetime – Morten Hekkvang Feb 04 '18 at 10:35
  • Sorry about the edits and bad formatting of text... I’m on a smartphone.. – Morten Hekkvang Feb 04 '18 at 10:39
  • Another thing. If you have control over the clients, maybe the timestamp send is configurable? – Morten Hekkvang Feb 04 '18 at 10:40

1 Answers1

0

This is one way, using pandas:

import pandas as pd

d = '2018-02-04T07:49:36.867Z'

d_pd = pd.to_datetime(d)  # Timestamp('2018-02-04 07:49:36.867000')
d_str = d_pd.strftime('%Y%m%d %T.%f')[:-3]  # '20180204 07:49:36.867'
jpp
  • 159,742
  • 34
  • 281
  • 339
  • Thanks for the reply, i did also the same by removing the unwanted strings and then changing the timestamp to datetime. – Mr. Confused Feb 05 '18 at 08:01