How to simplify list of dictionaries by condition?

Question

I have list of dictionaries of date objects:

{ "begin" :date object1, "end" : date object2 }
....
{ "begin" :date object3, "end" : date object4 }

I want to simplify this list by condition:

if cur.end == next.begin - datetime.timedelta(seconds=1))
    cur.end = next.end
    delete next

How to do it ?

Answer 1

I use integers instead of datetime objects to keep the code simple, but it's trivial to change it for your needs. Deleting items from a collection while iterating it will break the iterator, so you either use a second list (could be memory intensive) or you replace elements instead of deleting them. Since your elements are dictionaries I thought it's safe to replace the element to be deleted with None and at the end just filter out the Nones.

l=[
{ "b" : 1, "e" : 2},
{ "b" : 3, "e" : 5},
{ "b" : 6, "e" : 7},
{ "b" : 10, "e" : 12},
{ "b" : 13, "e" : 20}
]

for i in xrange(len(l) - 1):
    cur = l[i]
    if not cur:
        continue
    next = l[i + 1]
    if cur["e"] == next["b"] - 1:
        cur["e"] = next["e"]
        l[i+1] = None

l = filter(None, l)
print l

Since the first version doesn't completely fulfill your needs, repeat the steps until no more time intervals can be merged. It's inefficient and possibly the most non-pythonic piece of code today, but it does the job.

changed = True
while changed:
    changed = False
    l = filter(None, l)
    for i in xrange(len(l) - 1):
        cur = l[i]
        if cur is None:
            continue
        next = l[i + 1]
        if cur["e"] == next["b"] - 1:
            cur["e"] = next["e"]
            l[i+1] = None
            changed = True

l = filter(None, l)
print l

Answer 2

Like explained in the other answer, you should not delete an element from a list while iterating over it, it can lead to lots of issues. Another method that creates a completely new list would be -

import datetime
lisdic = [] #list of dictionaries
prev = None
result = []
for i in lisdic:
    if not prev:
        prev = i
    elif prev['end'] == i['begin'] - datetime.timedelta(seconds=1):
        prev['end'] = i['end']
    else:
        result.append(prev)
        prev = i
if prev:
    result.append(prev)

This would also handle similar intervals across multiple dictionaries (an Example being the first 3 dictionaries in the list in DEMO below).

Demo -

>>> import datetime
>>> lisdic = [{"begin":datetime.datetime(2015,10,2,10,0,0),"end":datetime.datetime(2015,10,2,10,30,0)},
... {"begin":datetime.datetime(2015,10,2,10,30,1),"end":datetime.datetime(2015,10,2,11,0,0)},
... {"begin":datetime.datetime(2015,10,2,11,0,1),"end":datetime.datetime(2015,10,2,12,0,0)},
... {"begin":datetime.datetime(2015,10,3,10,0,0),"end":datetime.datetime(2015,10,3,10,30,0)},
... {"begin":datetime.datetime(2015,10,3,11,0,0),"end":datetime.datetime(2015,10,3,11,30,0)},
... {"begin":datetime.datetime(2015,10,4,12,0,0),"end":datetime.datetime(2015,10,2,12,10,0)}]
>>> prev = None
>>> result = []
>>> for i in lisdic:
...     if not prev:
...         prev = i
...     elif prev['end'] == i['begin'] - datetime.timedelta(seconds=1):
...         prev['end'] = i['end']
...     else:
...         result.append(prev)
...         prev = i
...
>>>
>>> if prev:
...     result.append(prev)
...
>>> pprint.pprint(result)
[{'begin': datetime.datetime(2015, 10, 2, 10, 0),
  'end': datetime.datetime(2015, 10, 2, 12, 0)},
 {'begin': datetime.datetime(2015, 10, 3, 10, 0),
  'end': datetime.datetime(2015, 10, 3, 10, 30)},
 {'begin': datetime.datetime(2015, 10, 3, 11, 0),
  'end': datetime.datetime(2015, 10, 3, 11, 30)},
 {'begin': datetime.datetime(2015, 10, 4, 12, 0),
  'end': datetime.datetime(2015, 10, 2, 12, 10)}]