PyTorch mapping operators to functions

Question

What are all the PyTorch operators, and what are their function equivalents?

Eg, is a @ b equivalent to a.mm(b) or a.matmul(b)?

I'm after a canonical listing of operator -> function mappings.

I'd be happy to be given a PyTorch documentation link as an answer - my googlefu couldn't track it down.

Even though it is not that clear as a Doc Link, here are the actual definitions of the operators i.e. `+` and `__add__`: https://github.com/pytorch/pytorch/blob/fa8044d92f8a77b8008ca5e295a341abb9d26f13/torch/tensor.py#L292(This is not the current version, I haven't found such a clear definition in the current version, but I guess there hasn't changed too much) So you can check what is called then. `@` is defined by `__matmul__`. For the rest you check on [this python docs site](https://docs.python.org/2/library/operator.html) Hope this is helpful. — MBT, Nov 19 '18 at 10:31
@blue-phoenox Given those two links, how do I deduce that `@` -> `matmul`? — Tom Hale, Nov 19 '18 at 10:54
Check this and check this: https://stackoverflow.com/questions/27385633/what-is-the-symbol-for-in-python And check for the `__matmul__` function in the given (first) link. — MBT, Nov 19 '18 at 10:56

Tom Hale · Answer 1 · 2018-11-19T11:23:30.170

13

The Python documentation table Mapping Operators to Functions provides canonical mappings from:

operator -> __function__()

Eg:

Matrix Multiplication        a @ b        matmul(a, b)

Elsewhere on the page, you will see the __matmul__ name as an alternate to matmul.

The definitions of the PyTorch __functions__ are found either in:

You can look up the documentation for the named functions at:

https://pytorch.org/docs/stable/torch.html?#torch.<FUNCTION-NAME>

edited Nov 19 '18 at 11:23

answered Nov 19 '18 at 11:17

Tom Hale

40,825
36
187
242

what is `x.mm(y)` then? – Charlie Parker Aug 27 '20 at 17:46
1

@CharlieParker [`.mm()` is a non-broadcasting .matmul()](https://pytorch.org/docs/stable/generated/torch.mm.html?highlight=mm#torch.mm) – Tom Hale Aug 28 '20 at 17:09

score 4 · Answer 2 · answered Nov 19 '18 at 11:08

This defines tensor operations for 0.3.1 (it does also contain the definitions of the other operators): https://pytorch.org/docs/0.3.1/_modules/torch/tensor.html

The code for the current stable has been rearranged (I guess they do more in C now), but since the behaviour of matrix multiplication hasn't changed, I think it is save to assume that this is still valid.

See for the definition of __matmul__:

def __matmul__(self, other):
    if not torch.is_tensor(other):
        return NotImplemented
    return self.matmul(other)

and

def matmul(self, other):
    r"""Matrix product of two tensors.

    See :func:`torch.matmul`."""
    return torch.matmul(self, other)

The operator @ was introduced with PEP 465 and is mapped to __matmul__.

See also here for this:
What is the '@=' symbol for in Python?

Cheers! FYI, I think I found the [canonical answer](https://stackoverflow.com/a/53373484/5353461) — Tom Hale, Nov 19 '18 at 11:24

PyTorch mapping operators to functions

2 Answers2

Linked