2

I need to test the performance of code that will be objective and roughly the same across all machines. Timing code does not work since it's tied to your or mine machine specs, but counting instructions issued by a CPU does (with minor differences).

I can use strace in Linux, but my god its slow and I just want total not individual calls.

Say:

def foo(bar):
    for i in range(bar):
        print(i)

foo(10)

This will execute at different speeds on different machines (bear with me, imagine a more complicated algorithm). But the amount of operation done is the same, 10 ios. This is important because if you have a faster computer you won't notice a millisecond that might take 5 seconds on my machine.

Is there a way to count # of CPU instructions done since in Python?

I'm asking because I want to know if a refactor will 2x my CPU instructions.

Thank you.

The_Matrix
  • 85
  • 7
  • https://stackoverflow.com/questions/1557571/how-do-i-get-time-of-a-python-programs-execution – B.Quinn Dec 21 '21 at 05:06
  • @B.Quinn That gives me the time, not operations. Time will be different across machines, my testing environment is much faster than my laptop, but the operations are the same. Does Python have a module to count CPU operations or am I forced to port it from C. – The_Matrix Dec 21 '21 at 05:09
  • You can use the Python profiler cProfile to measure CPU time and additionally how much time is spent inside each function and how many times each function is called. This is very useful if you want to improve performance of your script without knowing where to start. This answer to another Stack Overflow question is pretty good. It's always good to have a look in the documentation too. -- This is an awnser from the link, kind of old but it seems possible to do, but I also read from that post that time() is usual "standard" you can use to find any performance gain then scale from there. – B.Quinn Dec 21 '21 at 05:31
  • You also might be able to use cython and call a C function, might be worth looking into. – B.Quinn Dec 21 '21 at 05:43
  • 2
    @B.Quinn Thank you for your help. I searched around and found a c library papi, and a python wrapper pypapi. It allows you to get total instructions issued from a CPU, but is only works on few cpu like Intel (and I'm on AMD) so the perfect solution doesn't work for me :( Does anyone know a portable way of getting instructions executed by a CPU inside Python? – The_Matrix Dec 21 '21 at 07:20
  • 3
    CPU instruction count is not an effective measure of performance. It doesn't consider I/O costs, or cache effects, or pipelining, or the different latencies and throughputs of different instructions, or all sorts of other effects. – user2357112 Dec 21 '21 at 07:26
  • Even for computations, the amount of instruction is generally not a good metric. CISC processors generally use less instructions can RISC ones while not being clearly faster or slower. Nowadays, instructions are executed in *parallel* and out-of-order by processor cores. Dependencies matters much more than the number of instructions. Not to mentions the impact of memory hierarchy at pointed out by @user2357112supportsMonica. For a deep analysis you need to use specific hardware counters. There are no portable counters. However, `perf` on Linux standardize some of them (see VTune on Windows). – Jérôme Richard Dec 21 '21 at 17:36

1 Answers1

-1

You can use the Python profiler cProfile

$ python -m cProfile euler048.py

1007 function calls in 0.061 CPU seconds

Ordered by: standard name
ncalls  tottime  percall  cumtime  percall 
filename:lineno(function)
    1    0.000    0.000    0.061    0.061 <string>:1(<module>)
    1000    0.051    0.000    0.051    0.000 euler048.py:2(<lambda>)
    1    0.005    0.005    0.061    0.061 euler048.py:2(<module>)
    1    0.000    0.000    0.061    0.061 {execfile}
    1    0.002    0.002    0.053    0.053 {map}
    1    0.000    0.000    0.000    0.000 {method someMethod}
    1    0.000    0.000    0.000    0.000 {range}
    1    0.003    0.003    0.003    0.003 {sum}

excerpt from the previous question I linked, hope this helps

B.Quinn
  • 77
  • 1
  • 7