2

I want a similar function as in https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.circulant.html to create a circulant matrix using PyTorch. I need this as a part of my Deep Learning model (in order to reduce over-parametrization in some of my Fully Connected layers as suggested in https://arxiv.org/abs/1907.08448 (Fig.3))

The input of the function shall be a 1D torch tensor, and the output should be the 2D circulant matrix.

rkosegi
  • 14,165
  • 5
  • 50
  • 83
Mevan Ekanayake
  • 21
  • 1
  • With how strides in tensors work, you can't construct a circulant matrix with a view (see [this discussion](https://discuss.pytorch.org/t/making-a-circulant-matrix-from-a-given-vector-in-a-memory-efficient-way/20908)). However you *can* make a copy, see [this thread](https://stackoverflow.com/questions/69723873/create-array-tensor-of-cycle-shifted-arrays/69725153#69725153). – Ivan Nov 03 '21 at 06:59
  • Does this answer your question? [Create array/tensor of cycle shifted arrays](https://stackoverflow.com/questions/69723873/create-array-tensor-of-cycle-shifted-arrays) – Tomerikoo Jan 12 '22 at 22:14

3 Answers3

1

You can make use of unfold to extract sliding windows. But to get the correct order you need to flip (later unflip) the tensors, and first concatenate the flipped tensor to itself.

circ=lambda v:torch.cat([f:=v.flip(0),f[:-1]]).unfold(0,len(v),1).flip(0)
flawr
  • 10,814
  • 3
  • 41
  • 71
0

Here is a generic function for pytorch tensors, to get the circulant matrix for one dimension. It's based on unfold and it works for 2d circulant matrix or high-dimension tensors.

def circulant(tensor, dim):
    """get a circulant version of the tensor along the {dim} dimension.
    
    The additional axis is appended as the last dimension.
    E.g. tensor=[0,1,2], dim=0 --> [[0,1,2],[2,0,1],[1,2,0]]"""
    S = tensor.shape[dim]
    tmp = torch.cat([tensor.flip((dim,)), torch.narrow(tensor.flip((dim,)), dim=dim, start=0, length=S-1)], dim=dim)
    return tmp.unfold(dim, S, 1).flip((-1,))

Essentially, this is a PyTorch version of scipy.linalg.circulant and works for multi-dimension tensors.

Also a similar question: Create array/tensor of cycle shifted arrays

htd
  • 1
  • 1
0

Based on this answer

n = len(vec)
arr = vec.repeat((n,1))
dim = 0 # all the rows
output = list(map(torch.roll, torch.unbind(arr, dim), tuple(range(n)))))
tensor_output = torch.stack(output)

Another method, based on this answer:

rolls = torch.arange(n)
vec[(torch.arange(n)[:,None]+rolls) % n]

The second method has been tested on 2d tensors.

YScharf
  • 1,638
  • 15
  • 20