How to do a FIFO push-operation for rows on Pandas dataframe in Python?

Question

I need to maintain a Pandas dataframe with 500 rows, and as the next row becomes available I want to push that new row in and throw out the oldest row from the dataframe. e.g. Let's say I maintain row 0 as newest, and row 500 as oldest. When I get a new data, I would push data to row 0, and it will shift row 0 to row 1, and so on until it pushes row 499 to row 500 (and row 500 gets deleted).

Is there a way to do such a FIFO operation on Pandas? Thanks guys!

1. do you mean, that you read data from a fifo? 2. using dataframe as a queue is very inefficient, because it takes O(N) time to insert/remove a row in your scenario. Just use a queue. — Eli Korvigo, Dec 28 '16 at 10:45
You could `df.shift(1); df.loc[0] = new_row`. That'll be slow tho. — Zero, Dec 28 '16 at 10:48
@JohnGalt, yes sir! That is exactly what I was looking for. Thanks a lot. — rrlamichhane, Dec 28 '16 at 11:06

rrlamichhane · Accepted Answer · 2022-01-13T23:30:47.473

7

@JohnGalt posted an answer to this on the comments. Thanks a lot. I just wanted to put the answer here just in case if people are looking for similar information in the future.

df.shift(1) df.loc[0] = new_row

df.shift(n) will shift the rows n times, filling the first n rows with na and getting rid of last n rows. The number of rows of df will not change with df.shift.

I hope this is helpful.

edited Jan 13 '22 at 23:30

answered Dec 30 '16 at 20:22

rrlamichhane

1,435
2
18
34

How to do a FIFO push-operation for rows on Pandas dataframe in Python?

1 Answers1

Linked