62

I feel embarrassed asking this, but how do you adjust a single value within a tensor? Suppose you want to add '1' to only one value within your tensor?

Doing it by indexing doesn't work:

TypeError: 'Tensor' object does not support item assignment

One approach would be to build an identically shaped tensor of 0's. And then adjusting a 1 at the position you want. Then you would add the two tensors together. Again this runs into the same problem as before.

I've read through the API docs several times and can't seem to figure out how to do this. Thanks in advance!

LeavesBreathe
  • 1,010
  • 2
  • 9
  • 17

6 Answers6

72

UPDATE: TensorFlow 1.0 includes a tf.scatter_nd() operator, which can be used to create delta below without creating a tf.SparseTensor.

This is actually surprisingly tricky with the existing ops! Perhaps somebody can suggest a nicer way to wrap up the following, but here's one way to do it.

Let's say you have a tf.constant() tensor:

c = tf.constant([[0.0, 0.0, 0.0],
                 [0.0, 0.0, 0.0],
                 [0.0, 0.0, 0.0]])

...and you want to add 1.0 at location [1, 1]. One way you could do this is to define a tf.SparseTensor, delta, representing the change:

indices = [[1, 1]]  # A list of coordinates to update.

values = [1.0]  # A list of values corresponding to the respective
                # coordinate in indices.

shape = [3, 3]  # The shape of the corresponding dense tensor, same as `c`.

delta = tf.SparseTensor(indices, values, shape)

Then you can use the tf.sparse_tensor_to_dense() op to make a dense tensor from delta and add it to c:

result = c + tf.sparse_tensor_to_dense(delta)

sess = tf.Session()
sess.run(result)
# ==> array([[ 0.,  0.,  0.],
#            [ 0.,  1.,  0.],
#            [ 0.,  0.,  0.]], dtype=float32)
mrry
  • 125,488
  • 26
  • 399
  • 400
  • Thank you immensely. I agree with you that a function that can do this internally with more efficiency would be useful! – LeavesBreathe Jan 09 '16 at 00:03
  • do you know how that handles values with the same index? – dtracers Jul 06 '16 at 18:21
  • nvm it does not handle that well at all... Do you know how to do this in the case of multiple indexes of the same value? – dtracers Jul 06 '16 at 21:01
  • 1
    @dtracers: I believe those two other questions are relevant: http://stackoverflow.com/questions/39157723/how-to-do-slice-assignment-in-tensorflow and http://stackoverflow.com/questions/39045797/conditional-assignment-of-tensor-values-in-tensorflow – Soravux Aug 30 '16 at 04:09
  • Could you rewrite this using the `scatter_nd` operation ? Would it be more efficient to use a variable for c in the first place ? – Mr_and_Mrs_D May 14 '17 at 18:43
  • @mrry why do you need `c`? won't `tf.sparse_tensor_to_dense(delta)` already have 1 at `[1,1]` and 0 everywhere else? – Nimrod Morag Jul 29 '18 at 06:01
8

How about tf.scatter_update(ref, indices, updates) or tf.scatter_add(ref, indices, updates)?

ref[indices[...], :] = updates
ref[indices[...], :] += updates

See this.

lmiguelvargasf
  • 63,191
  • 45
  • 217
  • 228
John Liu
  • 109
  • 1
  • 6
  • 3
    it's only valid if the `ref` is a Variable. – Sonal Gupta Jun 26 '17 at 21:23
  • That restriction is actually more fundamental than it looks, if you see TF as a restricted (in terms of available recursion schemes), scalable runtime of a mostly-pure lazy functional language. Then you can see that the difficulty of efficiently updating a (pure) tensor is essentially the same as the one you encounter in updating a purely functional data structure. Without that level of purity things wouldn't scale easily. – mnish Aug 01 '17 at 07:33
4

I feel like the fact that tf.assign, tf.scatter_nd, tf.scatter_update functions only work on tf.Variables is not stressed enough. So there it is.

And in later versions of TensorFlow (tested with 1.14), you can use indexing on a tf.Variable to assign values to specific indices (again this only works on tf.Variable objects).

v = tf.Variable(tf.constant([[1,1],[2,3]]))
change_v = v[0,0].assign(4)
with tf.Session() as sess:
  tf.global_variables_initializer().run()
  print(sess.run(change_v))
thushv89
  • 10,865
  • 1
  • 26
  • 39
1

tf.scatter_update has no gradient descent operator assigned and will generate an error while learning with at least tf.train.GradientDescentOptimizer. You have to implement bit manipulation with low level functions.

sentence
  • 8,213
  • 4
  • 31
  • 40
johannes
  • 11
  • 1
1

If you want to replace certain indices, I would create a boolean tensor mask and a broadcasted tensor with the new values at the correct positions. Then use 

new_tensor = tf.where(boolen_tensor_mask, new_values_tensor, old_values_tensor)
Patrick von Platen
  • 82
  • 5
1

If you want to add 1 to element [2,0] (for example) of your tensor v (make sure your tensor is Variable), simply write:

v[2,0].assign(v[2,0]+1)
Suraj Rao
  • 29,388
  • 11
  • 94
  • 103
Deano_McSmith
  • 11
  • 2