5

I read the following sentence in the TensorFlow documentation:

With the exception of tf.Variable, the value of a tensor is immutable, which means that in the context of a single execution tensors only have a single value. However, evaluating the same tensor twice can return different values; for example that tensor can be the result of reading data from disk, or generating a random number.

Can someone elaborate a little bit on the "immutable" aspect of a Tensor?

  1. What is the "scope of the immutability" since evaluating a tensor twice could return different results?
  2. What does it mean "the context of a single execution"?
CrazyBrazilian
  • 1,030
  • 1
  • 11
  • 15

1 Answers1

7

Tensors, differently from variables, can be compared to a math equation.

When you say a tensor equals 2+2, it's value is not actually 4, it's the computing instructions that leads to the value of 2+2 and when you start a session an execute it, TensorFlow runs the computations needed to return the value of 2+2 and gives you the output. And because of the tensor beeing the computations, rather than the the result, a tensor is immutable

Now for your questions:

  1. By saying the tensor can be evaluated with different values it means that if you for example say that a tensor equals to a random number, when you run it different times, you will have different values (as the equation itself is a random one), but the value of the tensor itself as mentioned before, is not the value, is the steps that leads to it (in this case a random formula)

  2. The context of a single execution means that when you run a tensor, it will only output you one value. Think executing a tensor like applying the equation i mentioned. If i say a tensor equals random + 1, when you execute the tensor a single time, it will return you a random value +1, nothing else. But since the tensor contains a randomic output, if you run it multiple times, you will most likely get different values

Rodolfo Donã Hosp
  • 1,037
  • 1
  • 11
  • 23
  • Great explanation. Thanks ! Would it be more precise to say that the definition of a tensor does not change, meaning that if a tensor defined an equation x+y, its value is the equation and that is immutable. However, when evaluating that tensor, you might receive different values based on x and y? – CrazyBrazilian Oct 18 '18 at 17:23
  • Exactly, when you build a tensor, TensorFlow basicaly builds a math model to compute whatever you want your output to be, hence, it's immutability, because the value of the Tensor itself is not the result of what you set, it is the "what do i need to compute in order to output the result that the user told me to", and if you ask your tensor to compute any random factor, executing it multiple times may lead to multiple values, even with the value of the tensor itself beeing the same. – Rodolfo Donã Hosp Oct 18 '18 at 17:35
  • For a better visualization, you can try setting a tensor and simply printing it, you will notice that you won't get the value you expected, instead, you will see a reference to a tensor object. If you want the result of it you need to open a session and call your_tensor.run, and at this point, it's computations will be actually executed and the result of it will be displayed. One of the reasons it is a very fast library, because code don't get unnecessarily executed – Rodolfo Donã Hosp Oct 18 '18 at 17:37
  • Hope it helps you understand the concepts behind it! – Rodolfo Donã Hosp Oct 18 '18 at 17:39
  • It did help! Thanks a lot ! – CrazyBrazilian Oct 18 '18 at 19:08
  • I was wondering whether this explanation will hold for Pytorch as well. As per my understanding the pytorch tensors are mutable. For mutating we use "_" post-fixed. Please correct me if I'm wrong – Ashish Johnson Sep 05 '20 at 18:28