4

I found out something weird.

I defined two test functions as such:

def with_brackets(n=10000):
    d = dict()
    for i in range(n):
        d["hello"] = i

def with_setitem(n=10000):
    d = dict()
    st = d.__setitem__
    for i in range(n):
        st("hello", i)

One would expect the two functions to be roughly the same execution speed. However:

>>> timeit(with_brackets, number=1000)
0.6558860000222921

>>> timeit(with_setitem, number=1000)
0.9857697170227766

There is possibly something I missed, but it does seem like setitem is almost twice as long, and I don't really understand why. Isn't dict[key] = x supposed to call __setitem__?

(Using CPython 3.9)

Edit: Using timeit instead of time

Phobia
  • 43
  • 5
  • 1
    Don't use `time.time` for doing code timing, it's unreliable. Use `timeit`. – Mark Ransom May 19 '21 at 03:42
  • 1
    After running these computations with `n=10000000` something like 25 times, I have two notes 1) I am confident there is a time difference, and 2) writing `d.__setitem__("hello", n)` each time is even slower than your current method of setting `setitem` to `d.__setitem__` – Kraigolas May 19 '21 at 03:47
  • @Kraigolas I knew about the d.__setitem__ being slower, which is why I took it out of the loop, thinking it would be the reason, and that's why I was quite confused to see that it was much slower even without.. – Phobia May 19 '21 at 03:53
  • 1
    I tried `timeit` with your code, and I get similar results - `with_brackets` is about twice as fast as `with_setitem`. And I also thought that the brackets would be converted to a call to `__setitem__` internally, so I'm at a loss to explain it. – Mark Ransom May 19 '21 at 03:54
  • 1
    For CPython specifically, `d["hello"]` calls `PyObject_SetItem` which calls `dict_ass_sub` (through the pointer `mp_ass_subscript`) directly whereas `st = d.__setitem__; st("hello", n)` wraps that C call in a Python call (as metatoaster showed) which introduces further overhead. – YiFei May 19 '21 at 04:24

1 Answers1

6

Isn't dict[key] = x supposed to call __setitem__?

Strictly speaking, no. Running both your functions through dis.dis, we get (I am only including the for loop):

>>> dis.dis(with_brackets)
...
        >>   22 FOR_ITER                12 (to 36)
             24 STORE_FAST               3 (i)

  5          26 LOAD_FAST                0 (n)
             28 LOAD_FAST                1 (d)
             30 LOAD_CONST               1 ('hello')
             32 STORE_SUBSCR
             34 JUMP_ABSOLUTE           22
...

Vs

>>> dis.dis(with_setitem)
...
        >>   28 FOR_ITER                14 (to 44)
             30 STORE_FAST               4 (i)

  6          32 LOAD_FAST                2 (setitem)
             34 LOAD_CONST               1 ('hello')
             36 LOAD_FAST                0 (n)
             38 CALL_FUNCTION            2
             40 POP_TOP
             42 JUMP_ABSOLUTE           28
...

The usage of __setitem__ involves a function call (see the usage of CALL_FUNCTION and POP_TOP instead of just STORE_SUBSCR - that's the difference underneath the hood), and function calls do add some amount of overhead, so using the bracket accessor leads to more optimised opcode.

metatoaster
  • 17,419
  • 5
  • 55
  • 66
  • That would explain it. However, how does it work in the case of a custom class implementing \_\_setitem\_\_? Would it also perform faster? It would seem counter-intuitive, but I'm not really familliar with all this. – Phobia May 19 '21 at 03:59
  • 2
    Maybe? You will have to run some tests, even though the bytecode generated using the bracket accessor will also remain as `STORE_SUBSCR` as the compiler will not discriminate. However, given that the custom `__setitem__` will ultimately be called, it may result in additional overhead. – metatoaster May 19 '21 at 04:11