By looking through various Questions on this site I have found 3 workable methods of reading a 4 byte (32 bit unsigned little-endian) integer from a file. Namely:
1) myInt, = struct.unpack('<I', bytes)
2) myInt = struct.unpack('<I', bytes)[0]
3) myInt = sum(bytes[i] << (i*8) for i in range(4))
Which of these is best? I know to use unpack requires importing the struct module but what are the other pros or cons for any particular method.
Assuming for best you mean more efficient, I would say any of the first two.
As you can see from this micro benchmark the third is a lot worse:
>>> bytes=b'\x10\x11\x12\x13'
>>> import struct
>>> import timeit
>>> timeit.timeit('a,=struct.unpack("<I", bytes)', 'from __main__ import struct, bytes')
0.16049504280090332
>>> timeit.timeit('a=struct.unpack("<I", bytes)[0]', 'from __main__ import struct, bytes')
0.1881420612335205
>>> timeit.timeit('sum(bytes[i] << (i*8) for i in range(4))', 'from __main__ import bytes')
1.2574431896209717
Also the third one does not work in python2, while the first and the second do(so they are also more portable).
The third is also not as readable, while having a bit of knowledge of struct it's easy to understand the first two versions.
Even though the first is slightly faster I'd choose the second, because that comma alone is not easy to see if reading fast, while [0] states clearly that you're taking the first element.
Also note that the difference in speed is really minimal and will probably change in newer/older versions of python, so using the first one just for the sake of speed will not be much of an optimization.
Update:
To explain why sum is so much slower(and more...):
Take into account that in python integers are objects, just like any other.
So when you do 5 + 2 you create two integer objects, and perform the __add__ method.
So the addition does not take one machine instruction.
That's why the bitshift solution is much slower, it has to created intermediate objects and perform some method calls(which "costs", because the arguments have to be packed and unpacked by the interpreter).
You should not assume that what is efficient in C is efficient in python.
A golden rule to optimize code in CPython(note: CPython not python. I'm talking for the official implementation, and not alternatives such as PyPy, Jython etc.), is to do as much computation as possible at "C level". By "C level" I mean inside functions written in C.
In this case the "C function" is struck.unpack, which is better than the solution using sum(note: inside sum there is a "python level" loop which is slower than a "C level" loop).
An other example is map:
#python2
>>> import timeit
>>> L = ['1', '2', '3'] * 5
>>> timeit.timeit('map(int, L)', 'from __main__ import L')
5.549130916595459
>>> timeit.timeit('[int(x) for x in L]', 'from __main__ import L')
6.402460098266602
(here longer is the list, faster the map solution is with respect to the list-comprehension)
I think it could be instructive for you to see this answer of mine where I show how a pure-python O(n) algorithm gets beaton for any reasonable input size by a O(n logn) algorithm using "C level" loops[also note [senderle's answer].
On why this does not work in python2:
giacomo@jack-laptop:~$ python2
Python 2.7.3 (default, Aug 1 2012, 05:14:39)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> bytes='\x10\x11\x12\x13'
>>> import timeit
>>> timeit.timeit('sum(bytes[i] << (i*8) for i in range(4))', 'from __main__ import bytes')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/timeit.py", line 230, in timeit
return Timer(stmt, setup, timer).timeit(number)
File "/usr/lib/python2.7/timeit.py", line 195, in timeit
timing = self.inner(it, self.timer)
File "<timeit-src>", line 6, in inner
File "<timeit-src>", line 6, in <genexpr>
TypeError: unsupported operand type(s) for <<: 'str' and 'int'
In python2 files return strings and string elements are strings, so you can't do the shift operation. If it worked for you then you were using python3.
