Split off list of dicts by key value - as with pop?

Question

I have seen Python: remove dictionary from list and Splitting a list of dictionaries into several lists of dictionaries - but this question is slightly different.

Consider this working example (same in Python 2 or 3):

#!/usr/bin/env python
from __future__ import print_function

origarr = [
  { 'name': 'test01', 'type': 0, 'value': 42 },
  { 'name': 'test02', 'type': 0, 'value': 142 },
  { 'name': 'test03', 'type': 2, 'value': 242 },
  { 'name': 'test04', 'type': 2, 'value': 342 },
  { 'name': 'test05', 'type': 3, 'value': 42 },
]

print("origarr: {}".format(origarr))

lastdictelem = origarr.pop()

print("\nlastdictelem: {}".format(lastdictelem))
print("after pop, origarr: {}".format(origarr))

namestofilter = [ 'test01', 'test02' ]
newarr = []
for iname in namestofilter:
  # find the object having the name iname
  foundidx = -1
  for ix, idict in enumerate(origarr):
    if idict.get('name') == iname:
      foundidx = ix
      break
  if foundidx > -1:
    # remove dict object via pop at index, save removed object
    remdict = origarr.pop(foundidx)
    # add removed object to newarr:
    newarr.append(remdict)

print("\nafter namestofilter:")
print("newarr: {}".format(newarr))
print("origarr: {}".format(origarr))

Basically, mylist.pop() removes the last element from mylist as an object (here a dict), and returns it - then I can trivially insert it in a new array/list; this is illustrated by the first printout of this script:

$ python2 test.py 
origarr: [{'name': 'test01', 'type': 0, 'value': 42}, {'name': 'test02', 'type': 0, 'value': 142}, {'name': 'test03', 'type': 2, 'value': 242}, {'name': 'test04', 'type': 2, 'value': 342}, {'name': 'test05', 'type': 3, 'value': 42}]

lastdictelem: {'name': 'test05', 'type': 3, 'value': 42}
after pop, origarr: [{'name': 'test01', 'type': 0, 'value': 42}, {'name': 'test02', 'type': 0, 'value': 142}, {'name': 'test03', 'type': 2, 'value': 242}, {'name': 'test04', 'type': 2, 'value': 342}]

Now, what I would like to do, is define an array with values for the name key in a dict (say, namestofilter = [ 'test01', 'test02' ]), and have those dicts removed from the orriginal array/list, and put into a new array/list (as .pop() would do with a single element and an object reference).

Since pop removes the item at a specific index and returns it, the above code does exactly that - and works:

...
after namestofilter:
newarr: [{'name': 'test01', 'type': 0, 'value': 42}, {'name': 'test02', 'type': 0, 'value': 142}]
origarr: [{'name': 'test03', 'type': 2, 'value': 242}, {'name': 'test04', 'type': 2, 'value': 342}]

... but I was wondering - is there a more compact way of doing that, other than "manually" for-looping through the two arrays, and calling .pop()/.append() individually (as done in the example)?

Why not just add to two new lists instead of doing a `pop` on the original (which can be up to `O(N)`)? — meowgoesthedog, Jan 24 '19 at 10:09
Thanks @meowgoesthedog - was not quite aware of `O(N)` for `pop`. As to why not two new lists - I guess, because my problem, semantically, is "remove some items from one list, then put them in another", and as I'd intend to use both lists afterwards, I didn't even think of two new lists; though I guess, they are a viable solution too (all I'd need to do is reassign the corresponding new one to the original variable name). — sdaau, Jan 24 '19 at 10:17

score 1 · Accepted Answer · answered Jan 24 '19 at 10:41

I'm not sure is there a way to it compact - probaly not.

But you can simpify code a little bit and also don't spend O(n) for each .pop:

origarr = [
  { 'name': 'test01', 'type': 0, 'value': 42 },
  { 'name': 'test02', 'type': 0, 'value': 142 },
  { 'name': 'test03', 'type': 2, 'value': 242 },
  { 'name': 'test04', 'type': 2, 'value': 342 },
  { 'name': 'test05', 'type': 3, 'value': 42 },
]

namestofilter = set([ 'test01', 'test02' ]). # could be a list as in question
print("origarr: {}".format(origarr))

lastdictelem = origarr.pop()

print("\nlastdictelem: {}".format(lastdictelem))
print("after pop, origarr: {}".format(origarr))

shift = 0
newarr = []
for ix, idict in enumerate(origarr):
    if idict['name'] in namestofilter:
        shift += 1
        newarr.append(idict)
        continue
    origarr[ix-shift] = origarr[ix]
origarr = origarr[:-shift]  # perhaps it is a slicing O(n) copy overhead 

print("\nafter namestofilter:")
print("newarr: {}".format(newarr))
print("origarr: {}".format(origarr))

Output:

origarr: [{'name': 'test01', 'type': 0, 'value': 42}, {'name': 'test02', 'type': 0, 'value': 142}, {'name': 'test03', 'type': 2, 'value': 242}, {'name': 'test04', 'type': 2, 'value': 342}, {'name': 'test05', 'type': 3, 'value': 42}]

lastdictelem: {'name': 'test05', 'type': 3, 'value': 42}
after pop, origarr: [{'name': 'test01', 'type': 0, 'value': 42}, {'name': 'test02', 'type': 0, 'value': 142}, {'name': 'test03', 'type': 2, 'value': 242}, {'name': 'test04', 'type': 2, 'value': 342}]

after namestofilter:
newarr: [{'name': 'test01', 'type': 0, 'value': 42}, {'name': 'test02', 'type': 0, 'value': 142}]
origarr: [{'name': 'test03', 'type': 2, 'value': 242}, {'name': 'test04', 'type': 2, 'value': 342}]

Split off list of dicts by key value - as with pop?

1 Answers1