6

When it comes to optimization of Python code (or any code), it often ends up coming down to profiling to optimize bottlenecks or slow functions. However, when optimizing these areas of code, are there any use cases for using pure Python bytecode inline? I know you can do this through the use of the compile built-in function and compiler module.

Suraj Jain
  • 4,463
  • 28
  • 39
IT Ninja
  • 6,174
  • 10
  • 42
  • 65

2 Answers2

7

... are there any use cases for using pure Python bytecode inline?

Yes. Sometimes you can handroll a little faster code than Python normally generates on its own.

Also, you can gain access to the loop induction variables for list comprehensions.

Here are some links to get you started: https://www.google.com/search?q=python+bytecode+hacks

If you would like to make the bytecode manipulations programmatically, here is an optimization recipe that shows how to go about it: Decorator for BindingConstants at compile time

That said, if you care about speed, the simplest speed-up is often to run pypy instead of cpython if your application allows it.

Raymond Hettinger
  • 216,523
  • 63
  • 388
  • 485
4

No. The source code is compiled to bytecode only once, when the module is first loaded. The bytecode is what is interpreted at runtime. So even if you could put bytecode inline into your source, it would at most only affect the startup time of the program by reducing the amount of time Python spent converting the source code into bytecode. It wouldn't actually change how fast that code runs. For instance, a "pure bytecode" version of a loop that runs 1000 times wouldn't run any faster than the same loop written in Python source code. See this similar question and also this one.

compile and compiler exist to let you dynamically create new executable source code from strings while the program is running. This isn't for a performance benefit, it's just because there's no other way to do it. When you run a Python program, Python compiles its source code to bytecode. But if you want to dynamically create new functions that aren't directly present in the source code (e.g., by mixing and matching code fragments, or allowing users to type in code while the program is running), you need a way to compile them on the fly, and that's what compile is for. (This is actually not a common need, so those functions aren't used that often.)

Community
  • 1
  • 1
BrenBarn
  • 242,874
  • 37
  • 412
  • 384
  • 2
    You're not going to write the *same* bytecode Python would, just like you wouldn't use inline assembler to write the same machine code your C compiler would. The point is to write better bytecode. – user2357112 Jan 06 '14 at 01:10
  • @user2357112: Although the answer below suggests that may be theoretically possible, it's not possible by using `compile`, because `compile` just compiles Python source code from a string. If you could get better bytecode by passing a string to `compile`, you could get the same bytecode by just putting that source code in your program directly. – BrenBarn Jan 06 '14 at 01:18
  • True, but he also mentions the `compiler` module, which can be used for constructing code objects more sophisticated than `compile` normally gives. Of course, the simpler way would be to create `types.CodeType` objects directly, if you like writing python bytecode by hand. The python vm is a pretty neat stack-oriented beast. – Perkins Sep 06 '15 at 08:05