2

I have been learning different ways of performing the same task as well as perform the same task in the same way just written differently. I can understand the time it takes to execute code that uses different methods can have a different amount of time taken to execute but I do not understand why code that performs the same task in the same method but just written differently can have a total execution time that is not on average the same.

For example.

I can run:

for sub in original_list:
    if sub[0] in to_check:
        new_list.append(sub)

in a section of my code and my average execution time is around 2.6 ms

However if I change the code to a one liner like this:

new_list = [sub for sub in original_list if sub[0] in to_check]

the execution time is on average 2.4 ms.

As I understand it both do the same operation so this brough up a few questions.

  1. Where is the change in time coming from?
  2. Is it scaling? For example if I iterate over a very large list will the time difference grow even bigger or will the time difference stay around 0.10 ms off?
  3. or am I just very confused and the 2 operations are not as identical as I originally thought?

I am very interested to know if there is a reason for this...

Mike - SMT
  • 14,784
  • 4
  • 35
  • 79
  • The piece that you have written in `[s for s in list]` is called a **list comprehension** and they are in general as fast or faster than writing the loop out as in your first block. Google list comprehensions python and read up on them. – Shawn Mehan Apr 24 '17 at 20:22
  • may be this link will answer you question http://stackoverflow.com/questions/22108488/are-list-comprehensions-and-functional-functions-faster-than-for-loops – plasmon360 Apr 24 '17 at 20:23
  • You can also learn about the differences between the *comprehension* and a *generator* which will also reduce your memory requirements. – Shawn Mehan Apr 24 '17 at 20:23
  • So they are different and not doing the same thing as I suspected. I did not know it was somewhat different in how it edits a list vs a for loop. This still leaves the question is is scaling? Does it really matter if I use list comprehension or a for loop? when working with mass amounts of date will it make a difference? – Mike - SMT Apr 24 '17 at 20:30

1 Answers1

2

The list comprehension is faster because it is optimized c code running against python. In particular, consider the line:

 new_list.append(sub)

While iterating in the loop, every time python gets to that line it has to:

  1. lookup new_list in the locals() namespace
  2. lookup the append method on that object (hash lookup!)
  3. call the append function

That "dot lookup" for .append is being done every time compared to the list comprehension where it knows from the structure that it's appending.

Phil Cooper
  • 5,747
  • 1
  • 25
  • 41
  • so is the list comprehension not performing all those task multiple times? The only code I have really spent any time learning is python over the 2 1/2 months so I am not sure what makes C run faster. or even how python knows it is going to run "optimized c code" Is this common to have small chunks of C inside of python? – Mike - SMT Apr 24 '17 at 20:33
  • @BaconTech, Don't think too much about the "C" part of my comment, think more of the python part. During each and every iteration of the for loop, python gets to the line and has to go out and do a dictionary lookup to find the "append" attribute of the "new_list" object. It is not cached in any way. These type of optimizations that PyPy can perform to outpace cPython code. – Phil Cooper Apr 24 '17 at 20:45
  • thanks for the clarification this make more sense now :) – Mike - SMT Apr 24 '17 at 21:02