Removing unwanted data from lists

Question

I have a list of varying length that I want to continuously up date with some new data. So basically I want to add a new data point and remove any data out of a set range. I have been playing around with this for a little bit now and haven't gotten anywhere that I can tell. I was trying to use this post as a reference, but apparently I don't under stand what is going on. Below is a code snippet that is an example of what I have tried.

for i in range(0,100):
    n.append(i)
    n = [x for x in n if not (x-n[-1]>10)]
    print len(n)

Ideally n would only have the last 10 data points contained in it at any given time during the for loop. I am sure that this is something basic that I am just not understanding, if you all could help me out I would really appreciate it. Thanks.

Edit: Example of the list n

[0]
[0, 1]
...
[89, 90, 91, 92, 93, 94, 95, 96, 97, 99]

"n would only have the last 10 data points" ... what exactly do you mean by last? most recent in terms of insertion? — Kaustubh Karkare, Oct 11 '12 at 18:53
can you give us an example of a list, and what you want to achieve. — root, Oct 11 '12 at 18:56
@KaustubhKarkare It would contain up to 10 of the most recent iterations (fewer before you hit 10), and then maintain a list of the 10 latest iteration numbers. At the end n would be [90, 91, ... 99]. — deadstump, Oct 11 '12 at 18:58
@KaustubhKarkare That worked great! Thanks! I am also looking into the n.pop(0) method. — deadstump, Oct 11 '12 at 19:14

score 4 · Answer 1 · answered Oct 11 '12 at 18:56

4

Why not just pop() the list every time you append something, if len>10? If I'm understanding the question right.

for i in range(0,100):
    n.append(i)
    if len(n)>10:
       n.pop(0)

answered Oct 11 '12 at 18:56

Colleen

23,899
12
45
75

This works as well as Kaustubh Karkare's answer (n[:] = n[1:]), but the out put shows the popped number as well (outside of the list, I am printing `n` so I can see it). Is this normal? – deadstump Oct 11 '12 at 19:13
code, please! Not sure what you mean-- e.g. outside the loop could be before or after. – Colleen Oct 11 '12 at 19:19
If I use your code with a `print n` after the `if` condition the last output looks like this `88` \n `[89, 90, ... 99]`. The `88` is the popped value, and the list is `n`. I was wondering if displaying the popped value is normal behavior. Thanks. – deadstump Oct 11 '12 at 19:33
1

Are you running this code in the interactive console? Because in that case, if the return values of statements aren't assigned to anything, they are displayed. – Kaustubh Karkare Oct 11 '12 at 19:40

score 4 · Accepted Answer · answered Oct 11 '12 at 18:56

4

Assuming you mean that n should contain only the latest 10 data points inserted, you want:

for i in range(0,100):
    n.append(i)
    if len(n)>10: n[:] = n[1:]
    print len(n) # will never go above 10

answered Oct 11 '12 at 18:56

Kaustubh Karkare

1,083
9
25

`n[:] = n[1:]` can be replaced by `n.pop(0)`, as Colleen said. – Kaustubh Karkare Oct 11 '12 at 18:57
what is the `n[:]` notation? Why not `n=n[1:]`? – Colleen Oct 11 '12 at 18:59
1

http://stackoverflow.com/questions/1207406/remove-items-from-a-list-while-iterating-in-python/1208792#1208792 – Kaustubh Karkare Oct 11 '12 at 19:01

score 1 · Answer 3 · answered Oct 11 '12 at 19:04

If I properly understand, you want to keep some variable number of elements in the list "n". Let's call that variable "m", so

for i in range(0,100):
    n.append(i)
    m = random.randint(1, 10)
    if len(n)>m:
        n = n[-m:]               # [-m:] defines the last m elements of n
    print len(n)

This should always print m in the end

root · Answer 4 · 2012-10-11T19:23:12.693

0

use deque with fast appends and pops: and from python 2.7 up you can set maxlen.

from collections import deque

>>> d=deque([])
>>> for i in range(10):
...     d.append(i)
...     if len(d) > 3: d.popleft()
... 
0
1
2
3
4
5
6
>>> d
deque([7, 8, 9])

Using maxlen:

>>> d = deque(range(5), maxlen=3)
>>> print d
deque([2, 3, 4], maxlen=3)
>>> print d.maxlen
3

edited Oct 11 '12 at 19:23

answered Oct 11 '12 at 19:16

root

76,608
25
108
120

Removing unwanted data from lists

4 Answers4