24

How do I convert a torch tensor to numpy?

Mateen Ulhaq
  • 24,552
  • 19
  • 101
  • 135
Gulzar
  • 23,452
  • 27
  • 113
  • 201

7 Answers7

39

copied from pytorch doc:

a = torch.ones(5)
print(a)

tensor([1., 1., 1., 1., 1.])

b = a.numpy()
print(b)

[1. 1. 1. 1. 1.]

Following from the below discussion with @John:

In case the tensor is (or can be) on GPU, or in case it (or it can) require grad, one can use

t.detach().cpu().numpy()

I recommend to uglify your code only as much as required.

Mateen Ulhaq
  • 24,552
  • 19
  • 101
  • 135
Gulzar
  • 23,452
  • 27
  • 113
  • 201
  • 1
    In my copy of torch better make that a.detach().cpu().numpy() – Lars Ericson Aug 13 '19 at 02:37
  • @LarsEricson why? – Gulzar Jan 26 '20 at 08:37
  • what would the complexity be to convert tensor to NumPy like this? – Sid Jan 04 '21 at 17:23
  • @Sid I believe `o(1)` in most cases, but not always. See https://github.com/pytorch/pytorch/blob/master/torch/csrc/utils/tensor_numpy.cpp and https://numpy.org/devdocs/reference/c-api/array.html#c.PyArray_SetBaseObject – Gulzar Jan 04 '21 at 17:54
  • @Gulzar the detach() is necessary to avoid computing gradients and the cpu() is necessary if the Tensor is in gpu memory – John Jan 23 '21 at 19:42
  • @John this is not necessary in the general case. – Gulzar Jan 23 '21 at 21:43
  • @Gulzar can you be more specific? What isn't necessary (there were 2 functions listed) and what do you mean by "in the general case?" consider this question/answer for more detail: https://stackoverflow.com/questions/63582590/why-do-we-call-detach-before-calling-numpy-on-a-pytorch-tensor – John Jan 25 '21 at 00:07
  • @John The usage of `detach` and `cpu` is understood. The question here was about converting a torch tensor to a numpy array. numpy arrays are always on cpu, and always don't have gradient calculation involved with them. If the torch tensor is on gpu or needs to be detached, that is beside the point of this question. So, *always* doing `.detach().cpu()` is overkill. Doing it when necessary is good, but this isn't the general case. – Gulzar Jan 25 '21 at 10:04
  • 1
    This is true, although I believe both are noops if unnecessary so the overkill is only in the typing and there's some value if writing a function that accepts a Tensor of unknown provenance. I apologize for misunderstanding your original question to Lars. To summarize, `detach` and `cpu` are not necessary in every case, but are necessary in perhaps the most common case (so there's value in mentioning them). `numpy` is necessary in every case but is often insufficient on its own. Any future persons should reference the question linked above or the pytorch documentation for more information. – John Jan 26 '21 at 14:34
11

You can try following ways

1. torch.Tensor().numpy()
2. torch.Tensor().cpu().data.numpy()
3. torch.Tensor().cpu().detach().numpy()
Ashiq Imran
  • 2,077
  • 19
  • 17
5

Another useful way :

a = torch(0.1, device='cuda')

a.cpu().data.numpy()

Answer

array(0.1, dtype=float32)

Gulzar
  • 23,452
  • 27
  • 113
  • 201
Eyshika
  • 1,041
  • 1
  • 11
  • 18
4

This is a function from fastai core:

def to_np(x):
    "Convert a tensor to a numpy array."
    return apply(lambda o: o.data.cpu().numpy(), x)

Possible using a function from prospective PyTorch library is a nice choice.

If you look inside PyTorch Transformers you will find this code:

preds = logits.detach().cpu().numpy()

So you may ask why the detach() method is needed? It is needed when we would like to detach the tensor from AD computational graph.

Still note that the CPU tensor and numpy array are connected. They share the same storage:

import torch
tensor = torch.zeros(2)
numpy_array = tensor.numpy()
print('Before edit:')
print(tensor)
print(numpy_array)

tensor[0] = 10

print()
print('After edit:')
print('Tensor:', tensor)
print('Numpy array:', numpy_array)

Output:

Before edit:
tensor([0., 0.])
[0. 0.]

After edit:
Tensor: tensor([10.,  0.])
Numpy array: [10.  0.]

The value of the first element is shared by the tensor and the numpy array. Changing it to 10 in the tensor changed it in the numpy array as well.

This is why we need to be careful, since altering the numpy array my alter the CPU tensor as well.

prosti
  • 42,291
  • 14
  • 186
  • 151
1

You may find the following two functions useful.

  1. torch.Tensor.numpy()
  2. torch.from_numpy()
Wasi Ahmad
  • 35,739
  • 32
  • 114
  • 161
1

Sometimes if there's "applied" gradient, you'll first have to put .detach() function before the .numpy() function. 

loss = loss_fn(preds, labels)
print(loss.detach().numpy())
Marius Johan
  • 394
  • 3
  • 13
0
x = torch.tensor([0.1,0.32], device='cuda:0')

x.detach().cpu().data.numpy()
KingLiu
  • 59
  • 4