Does single line loops save the run time for python code in comparison to multi-line loops?

Question

I want to know how to reduce the run time of my program. Are using single line loops more effcient than multi line?

#is this more efficient

total_data = [[arr1[j][i] for j in range(3)]+ arr2[i][0] for i in range(10000)]

#instead of

total_data = []
for i in range(10000):
    arr3 = []
    a2 = arr2[i][0]
    for j in range(3):
        arr3.append(arr1[j][i])
    total_data.append(arr3+a2)

Also, when calling a function, does using map save more time, instead of for loop?

#this

f1 = map(func1, var1, var2)
arr = map(func2, f2, var3, var4)

#instead of this

arr = []
for i in range(1000):
    f1 = func1(var1(i), var2(i))
    f2 = func2(f1(i))
    arr.append(f2, var3, var4)

My data set is large and each function run time is also considerable, so I want to reduce the time as much as possible. I want to know fundamentally whether increasing lines in python for the same loop increases the time.

Answer 1

The question of whether more lines of code imply less performance is flawed because neither less lines ≠ performance ≠ more lines. Performance would depend on how the Python code is parsed to the underlying C function calls that would execute it, and this is mostly an implementation detail of the parser/interpreter without a one-to-one equivalence with the amount of lines.

That said, here are some points to be accounted for when choosing loop vs list comprehension vs map.

• When the purpose of the code is to create a list of something then list comprehensions are usually faster than loops. AFAIK this has to do with the second doing calls to the append method and also because the incremental list construction may cause the list to resize, causing some memory reallocation overhead.

• Calling a python function is a slow operation regardless of where it is called. Quoting from this great answer: A function call needs to manipulate the stack, pushing the local frame onto it, creating a new frame, then clear it all up again when the function returns. One exception to this are functions coded in C, for example the ones from the operator module, which are slightly faster than the native Python counterparts.

That said I've made some profiling using code similar to this one and my the tests showed that the loop is the slowest option. Funnily enough the test also showed that map was slightly faster than list comprehension (in contrast with the answer linked). Code and results below:

ls = list(range(1000000))

def f(x):
    return x+1

def list_comprehension(ls):
    return [f(x) for x in ls]

def for_loop(ls):
    arr = []
    for x in ls:
        arr.append(f(x))
    return arr

def map_(ls):
    return list(map(f, ls))

if __name__ == "__main__":
    import cProfile
    for fn in [list_comprehension, for_loop, map_]:
        print('=' * 25)
        print("Profiling:", fn.__name__)
        print('=' * 25)
        pr = cProfile.Profile()
        for i in range(1000):
            pr.runcall(fn, ls)
        pr.create_stats()
        pr.print_stats()

# Output
=========================
Profiling: list_comprehension
=========================
         1000003000 function calls in 235.641 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1000000000  104.000    0.000  104.000    0.000 aa.py:5(f)
     1000    0.008    0.000  235.640    0.236 aa.py:8(list_comprehension)
     1000  131.632    0.132  235.632    0.236 aa.py:9(<listcomp>)
     1000    0.001    0.000    0.001    0.000 {method 'disable' of '_lsprof.Profiler' objects}


=========================
Profiling: for_loop
=========================
         2000002000 function calls in 410.884 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1000  242.083    0.242  410.883    0.411 aa.py:11(for_loop)
1000000000  107.727    0.000  107.727    0.000 aa.py:5(f)
1000000000   61.073    0.000   61.073    0.000 {method 'append' of 'list' objects}
     1000    0.001    0.000    0.001    0.000 {method 'disable' of '_lsprof.Profiler' objects}


=========================
Profiling: map_
=========================
         1000002000 function calls in 205.035 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1000  102.451    0.102  205.034    0.205 aa.py:17(map_)
1000000000  102.583    0.000  102.583    0.000 aa.py:5(f)
     1000    0.001    0.000    0.001    0.000 {method 'disable' of '_lsprof.Profiler' objects}

Anyway, the best thing to do in this cases is to write down all different versions and do the profiling yourself rather than relying in general assumptions.

Answer 2

Number of lines is absolutely irrelevant for the execution of the code. The py file is going to be read once, parsed and compiled. Thereon, the compiled code will be executed.

In other words, if you create a file with 10 GB of whitespace, yes, that will influence the parsing time, but even then the execution will not be affected.

The difference between your two variants is that one uses a list comprehension to create a list, whereas the other is creating an empty list and then filling it. My gut feeling is that a list comprehension is theoretically easier to optimise, but if you really care for the minute differences, you should profile it.

On the other hand, a small difference in execution time rarely justifies making the code look worse, so you should always go for the more elegant solution by default.

Answer 3

One-liners can be faster than longer code, but that's not a general rule.Some python functions like any(), all() etc. that allow for very compact code are optimized to be executed very quickly and will probably be faster than (for example) a couple of nested loops that do the same job. A compiler/interpreter doesn't really care how much text it has to process (unless your code consists of thousands of lines of code). The one liner is preferred ,because it saved space and make your code much more readable.