Minimising a python code footprint with nested loops

Question

I have a code that with two lists of dictionaries, and should make a final dictionary based on the comparing of two lists.

This is the code:

list_a = [
   {'e1':20},
   {'e1':11.2},
   {'e1':20.33},
   {'e1':19.34},
   {'e1':18.2}
 ]

list_b = [
   {'id': 1, 'e1':20, 'status':True},
   {'id': 2, 'e1':11.2, 'status':False},
   {'id': 3, 'e1':20.33, 'status':True}
]

mydict = {}
for b in list_b:
  mydict.setdefault(b['id'], {})
  if b['status']:
    for a in list_a:
        if b['e1'] - a['e1'] > 1:
            mydict[b['id']] = b['e1'] - a['e1']

Just wondering if you have any solution to make this code more efficient (both speed and space aspects)?

@J2015 Are you using any imports? This does not run for me as presented in python 3.7. — User1010, Mar 07 '21 at 12:15
Your code doesn't work, so it's currently impossible to make it more efficient (at least, not in the usual sense). — 9769953, Mar 07 '21 at 12:15
@EnderShadow8 a trade off between speed and space efficiency — J2015, Mar 07 '21 at 12:21
@User1010 Nothing to do with python version I guess, however mine is 3.7 as well — J2015, Mar 07 '21 at 12:22
I think the time complexity is about right, just micro optimisations then — EnderShadow8, Mar 07 '21 at 12:28
why do you care if you can remove 0.000004 s from this operation? Code works, it is kindof clear what it does. If you do not have any bottlenecks - use as is. — Patrick Artner, Mar 07 '21 at 12:48
There is https://codereview.stackexchange.com/ , which may be more appropriate for your question. — 9769953, Mar 07 '21 at 12:48
@PatrickArtner no-one mentioned 0.0000000004, certainly not the OP. I could imagine this is a toy example, and the actual use case is either called millions of times, or with much larger input dicts. — 9769953, Mar 07 '21 at 12:49
There is no speach of millions of things. For optimizing questions we need the size it runs in. Simple (and covered 100x) are using defaultdict(dict) instead of dict.setdefault. The code is clear and correct and plenty fast with the given data. Optimizing it is pointless. If you want to extend it to bigger inputs, we would need to know about those dimensions. — Patrick Artner, Mar 07 '21 at 12:53
[setdefault-vs-defaultdict-performance](https://stackoverflow.com/questions/38625608/setdefault-vs-defaultdict-performance) - — Patrick Artner, Mar 07 '21 at 12:58

score 0 · Answer 1 · answered Mar 07 '21 at 12:39

One way to shave off time is to avoid using mydict.setdefault,

import time

list_a = [
    {'e1': 20},
    {'e1': 11.2},
    {'e1': 20.33},
    {'e1': 19.34},
    {'e1': 18.2}
]

list_b = [
    {'id': 1, 'e1': 20, 'status': True},
    {'id': 2, 'e1': 11.2, 'status': False},
    {'id': 3, 'e1': 20.33, 'status': True}
]

time_original = 0
time_new = 0

for loop in range(1000000):
    time1 = time.time()
    mydict = {}
    for b in list_b:
        mydict.setdefault(b['id'], {})
        if b['status']:
            for a in list_a:
                if b['e1'] - a['e1'] > 1:
                    mydict[b['id']] = b['e1'] - a['e1']
    time2 = time.time()

    time_original += time2 - time1

    time3 = time.time()
    example_dict = {}
    for b in list_b:
        example_dict[b['id']] = {}
        if b['status']:
            for a in list_a:
                if b['e1'] - a['e1'] > 1:
                    example_dict[b['id']] = b['e1'] - a['e1']
    time4 = time.time()

    time_new += time4 - time3

print('Using mydict.setdefault(b[\'id\'], {})')
print(time_original / 1000000)
print(mydict)
print()
print('Using example_dict[b[\'id\']] = {}')
print(time_new / 1000000)
print(example_dict)

Results,

Using mydict.setdefault(b['id'], {})
2.1369829177856445e-06
{1: 1.8000000000000007, 2: {}, 3: 2.129999999999999}

Using example_dict[b['id']] = {}
2.096635103225708e-06
{1: 1.8000000000000007, 2: {}, 3: 2.129999999999999}

a shave-off of 0.04e-06 - that is 0.00000004 seconds .... seems kindof pointless? Beside that, using defaultdict(dict) would be even faster .... — Patrick Artner, Mar 07 '21 at 12:53
@PatrickArtner I agree the time saving is pointless however this a purely academic observation. defaultdict(dict) does not produce the same output, the length of the resulting dict would only be 2 not 3. — User1010, Mar 07 '21 at 13:07

Minimising a python code footprint with nested loops

1 Answers1