How to efficiently create nested loops and append the output (as a multi-dimensional array) from all the loops?

Question

I'm looking for an 'efficient' way in Python 3.x to create various nested loops, and then append the result of each inner loop (a multi-dimensional array).

For instance, the function model_A() has 3 parameters (a, b, c), and I want to enumerate all of the possibilities to test the model. The casual way is:

result_a = []
for a_value in a:
    result_a_b = []
    for b_value in b:
        result_a_b_c = []
        for c_value in c:
            result = model_A(a, b, c)
        result_a_b_c.append(result)
    result_a_b.append(result_a_b_c)
result_a.append(result_a_b)

I think there should be a way to 'efficiently' create the nested loop, and append the result, without having to create an empty list before each loop, and append the result at the end of each inner loop.

Thank you for your comment. Would you please elaborate a bit more for me? — lonelyhill, Jan 15 '19 at 13:49
It's difficult to understand what exactly you need to have done, but the `product` function from itertools may be what you need. Adding some examples of expected input and output would help verify. — Marcus, Jan 15 '19 at 13:50
Possible duplicate of [Get the cartesian product of a series of lists?](https://stackoverflow.com/questions/533905/get-the-cartesian-product-of-a-series-of-lists) — Georgy, Jan 15 '19 at 13:53
Hi Marcus, as in my example above, with 3 parameters to test, I have to create 3 empty list. When I get to the very inner loop, I perform some computations, and append the result from that inner loop to the outer loop another 3 times. Is there a better way to do this, in case I have, say, 100 parameters to test? — lonelyhill, Jan 15 '19 at 13:57
@lonelyhill I made some style edits in your question, but I noticed that your code will not work correctly, because you need to indent each of the 3 `.append()` lines (the `.append()` has to happen inside the corresponding loop). — Ralf, Jan 15 '19 at 14:06

score 2 · Accepted Answer · answered Jan 15 '19 at 13:55

2

Probably itertools.product() has what you need

for instance:

a = [1, 2]
b = [3, 4]
c = [5, 6]

for element in itertools.product(a,b,c):
    print(element)

Results in:

(1, 3, 5)
(1, 3, 6)
(1, 4, 5)
(1, 4, 6)
(2, 3, 5)
(2, 3, 6)
(2, 4, 5)
(2, 4, 6)

This might enable you to avoid the deeply nested for loops.

answered Jan 15 '19 at 13:55

Filipe Aleixo

3,924
3
41
74

I don't know if this is what the OP (@lonelyhill) asked for, since this does not return a multi-dimensional array. – Ralf Jan 15 '19 at 14:09
@Ralf Thank you for your follow up. I think I can make an empty multi-dimension list using a recursive function, then simply call `.append` to collect the result based on the indices generated from `itertools.product(a, b, c)`. – lonelyhill Jan 16 '19 at 05:29

score 1 · Answer 2 · answered Jan 15 '19 at 14:13

One small improvement can be had by using list comprehension (even though it is not so readable in my opinion in this case):

A_LIST = list(range(10))
B_LIST = list(range(10))
C_LIST = list(range(10))

def model_A(a, b, c):
    # mock implementation that does some calculation
    return a + b * c

def f1():
    # your version
    result_a = []
    for a_value in A_LIST:
        result_a_b = []
        for b_value in B_LIST:
            result_a_b_c = []
            for c_value in C_LIST:
                result = model_A(a_value, b_value, c_value)
                result_a_b_c.append(result)
            result_a_b.append(result_a_b_c)
        result_a.append(result_a_b)

    return result_a

def f2():
    # some improvement
    return [
        [
            [model_A(a_value, b_value, c_value)
             for c_value in C_LIST]
            for b_value in B_LIST]
        for a_value in A_LIST]

Comparing these 2 versions, list comprehension is slightly faster:

>>> f1() == f2()
True

>>> import timeit
>>> timeit.timeit('f()', 'from __main__ import f1 as f', number=10000)
1.905
>>> timeit.timeit('f()', 'from __main__ import f2 as f', number=10000)
1.503

Thank you so much for your comment. I guess the only drawback of `f2()` is that we still have to manually specify these `for` loops. But I greatly appreciate your idea! — lonelyhill, Jan 16 '19 at 05:32

How to efficiently create nested loops and append the output (as a multi-dimensional array) from all the loops?

2 Answers2