Questions tagged [einsum]
14 questions
1
vote
1 answer
Batch axes in numpy operations
Why are the "batch" axes always the leading axes in NumPy? I designed all my packages to use the trailing axes as batch axes because this seems more natural to me. Now I'm thinking about switching to NumPy's convention - just to make things more…
adtzlr
- 45
- 5
1
vote
1 answer
Numpy einsum getting crazy slow after certain problem size
I have the following benchmarking script
import numpy as np
import timeit
import matplotlib.pyplot as plt
n1, n2, n3, n4, n5, m = (101, 33, 1, 32, 2, 32)
def make_arrays(aOrder, bOrder, cOrder):
a = np.random.randn(n1, m) + 1j *…
Labello
- 323
- 3
- 15
1
vote
1 answer
How to add another dimension to an einsum operation?
Suppose the tensor and tensor1 are some calculated transformations of an input with the shapes provided in the code snippet. The einsum operation performs Einstein's summation to aggregate the results in a specific order.
import tensorflow as…
Ali Khalili
- 83
- 8
1
vote
0 answers
Speed up einsum for sparse tensors
I would like to perform the following batch of matrix multiplications
proj = torch.einsum('abi,aic->abc', A, B)
where A is an nxnxd tensor and B is an nxdxd tensor.
When n gets large ~50k, this operation becomes very slow.
However, A is actually…
Adam Gosztolai
- 242
- 1
- 3
- 14
1
vote
1 answer
Julia: ERROR: LoadError: MethodError: Cannot `convert` an object of type Expr to an object of type Symbol
I am trying to multiply two array via the Einsum package (uses Meta Programming). I get the following error if I use the @einsum macro with elements of a struct but not if I copy the element beforehand. Can someone explain?
using Einsum
struct…
Martin
- 50
- 5
1
vote
1 answer
Understanding fancy einsum equation
I was reading about attention and came across this equation:
import einops
from fancy_einsum import einsum
import torch
x = torch.rand((200, 10, 768))
y = torch.rand((20, 768, 64))
res = einsum("batch query_pos d_model, n_heads d_model d_head ->…
Sai Prashanth
- 144
- 2
- 11
1
vote
1 answer
Einsum for shapes of different sizes or ranks
I have two PyTorch tensors. One is rank three and the other is rank four. Is there a way to get it so that it produce the rank and shape of the first tensor? For instance in this cross-attention bit:
q = torch.linspace(1, 192, steps=192)
q =…
MScottWaller
- 3,321
- 2
- 24
- 47
0
votes
0 answers
tf.matmul vs tf.einsum for constructing density matrices from a set of Cholesky decomposed matrices
I am trying to construct a density matrix of shape 256x256 from a set of T matrices. These T matrices are all Cholesky-decomposed matrices. But I am not sure if the code is right or my approach is right; one of the reasons is that when I change from…
Dimitri
- 109
- 1
- 14
0
votes
0 answers
How to perform T-Product of two Tensors using einsum notation?
Tensor Product :
I want to take T-Product of two Tensors
$\matcal{A}, \matcal{B} \in C^{n \times n \times n}$
So, as you all know, T-Product is defined as,
$C = Fold(Batch_Circulant(\matcal{A} * Unfold(\matcal{B}))$
I am trying to solve this using…
Arima
- 1
0
votes
1 answer
Speedup with einsum and average over multiple arrays
Given two matrices A and B with 4 indices each, A[i,j,n_i,m], B[i,j,n_i,m], I need to perform the tensor product of them for the first two indices, while keeping the remaning two. For this task, I use einsum. Moreover, I need to average the result…
J.Agusti
- 161
- 11
0
votes
0 answers
rewrite einsum("bhwHW,bHWc->bhwc") without einstein notation
The example code of DDPM works well in python. But after I convert it into tensorflowjs model, and run it in web browser, this line failed with error.
tf.einsum("bhwHW,bHWc->bhwc", attn_score, v)
Error: Found duplicate axes in input component…
Mr.Wang from Next Door
- 13,670
- 12
- 64
- 97
0
votes
0 answers
the first calculation with torch.einsum is much slower
When I run several calculations with torch.einsum in a row, the first one is always much slower than the following calculations.
The following code and plot illustrates the problem:
import torch as tor
from timeit import default_timer as timer
N =…
Fynn
- 1
- 2
-1
votes
1 answer
Einsum multiply each row with every one for 3X3X3 array
Hello could someone please help me figure out how to use np.einsum to produce the below code's result. I have a (3,3,3) tensor and I will like to get this results which I got from using two for loops. the code I wrote to produce this output is…
mike talker
- 3
- 2