0

As example i use the following DF:

Index                   Temperature
2019-11-14 08:25:30.000 50.776897   
2019-11-14 08:25:40.000 49.671967
2019-11-14 08:25:50.000 48.729610
2019-11-14 08:26:00.000 48.022270
2019-11-14 08:26:10.000 47.502003
2019-11-14 08:43:40.000 87.997314  <-- here i want to split the DF into two DF's
2019-11-14 08:43:50.000 90.791519
2019-11-14 08:44:00.000 93.656128
2019-11-14 08:44:10.000 95.525215
2019-11-14 08:44:20.000 95.694481

So i use a large DF and i want to split the DF into small DF's when there is a jump in the timestamp index. Or in other words I would like to split the dataframe whenever the next timestep is larger than 10 seconds later.

Machavity
  • 30,841
  • 27
  • 92
  • 100
RvBenthem
  • 35
  • 4
  • 1
    did you have a look at [Pandas split DataFrame by column value](https://stackoverflow.com/q/33742588/10197418)? as for `value` this would mean the timedelta between subsequent timestamps. – FObersteiner Aug 05 '20 at 12:25

1 Answers1

0

I have added a group column to create different groups based on time. So when the index time difference is >10s a new group will be created.

Then i made a list of DF's and used a loop to go through the DF to split it,based on group number.

df['groups'] = (df.index.to_series().diff().dt.seconds > 10).cumsum()

list_of_dfs = []
for ct, data in df.groupby('groups'):
    list_of_dfs.append(data)

the result:

Index                   Temperature  groups
2019-11-14 08:25:30.000 50.776897    0
2019-11-14 08:25:40.000 49.671967    0
2019-11-14 08:25:50.000 48.729610    0
2019-11-14 08:26:00.000 48.022270    0
2019-11-14 08:26:10.000 47.502003    0

2019-11-14 08:43:40.000 87.997314    1
2019-11-14 08:43:50.000 90.791519    1
2019-11-14 08:44:00.000 93.656128    1
2019-11-14 08:44:10.000 95.525215    1
2019-11-14 08:44:20.000 95.694481    1
RvBenthem
  • 35
  • 4