Tensorflow 2 - tf.slice and its NumPy slice syntax incompatible behavior

Question

Question

Please confirm if the below is as designed and expected, or an issue of tf. slice, or a mistake in the usage of tf. slice. If a mistake, kindly suggest how to correct it.

Background

Introduction to tensor slicing - Extract tensor slices says Numpy-like slice syntax is an alternative of tf. slice.

Perform NumPy-like tensor slicing using tf. slice.
t1 = tf.constant([0, 1, 2, 3, 4, 5, 6, 7])
print(tf.slice(t1,
               begin=[1],
               size=[3]))
Alternatively, you can use a more Pythonic syntax. Note that tensor slices are evenly spaced over a start-stop range.
print(t1[1:4])

Problem

To Update the dark orange region.

TYPE = tf.int32
N = 4
D = 5
shape = (N,D)

# Target to update
Y = tf.Variable(
    initial_value=tf.reshape(tf.range(N*D,dtype=TYPE), shape=shape),
    trainable=True
)
print(f"Target Y: \n{Y}\n")
---
Target Y: 
<tf.Variable 'Variable:0' shape=(4, 5) dtype=int32, numpy=
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19]], dtype=int32)>

tf. slice does not work.

# --------------------------------------------------------------------------------
# Slice region in the target to be updated
# --------------------------------------------------------------------------------
S = tf.slice(      # Error "EagerTensor' object has no attribute 'assign'"
    Y,
    begin=[0,1],   # Coordinate (n,d) as the start point
    size=[3,2]     # Shape (3,2) -> (n+3, n+2) as the end point
)
print(f"Slice to update S: \n{S}\n")

# Values to set
V = tf.ones(shape=tf.shape(S), dtype=TYPE)
print(f"Values to set V: \n{V}\n")

# Assing V to S region of T
S.assign(V)
---
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-17-e5692b1750c8> in <module>
     24 
     25 # Assing V to S region of T
---> 26 S.assign(V)

AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'assign'

However, slice syntax works.

S = Y[
    0:3,           # From coordinate (n=0,d),   slice rows (0,1,2)  or 'size'=3 -> shape (3,?)
    1:3            # From coordinate (n=0,d=1), slice columns (1,2) or 'size'=2 -> shape (3,2)
]                  
print(f"Slice to update S: \n{S}\n")

# Values to set
V = tf.ones(shape=tf.shape(S), dtype=TYPE)
print(f"Values to set V: \n{V}\n")

# Assing V to S region of T
S.assign(V)
---
<tf.Variable 'UnreadVariable' shape=(4, 5) dtype=int32, numpy=
array([[ 0,  1,  1,  3,  4],
       [ 5,  1,  1,  8,  9],
       [10,  1,  1, 13, 14],
       [15, 16, 17, 18, 19]], dtype=int32)>

score 1 · Accepted Answer · answered May 04 '21 at 15:45

In my understanding, the above behavior is expected or not a bug at least. As the error said, there is no attribute called assign in tf. Tensor (EagerTensor for eager execution) but there is in tf. Variable. And generally, tf. slice returns a tensor as its output and thus it doesn't possess assign attribute.

AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'assign'

But when we do np like slicing and use it to modify the original tf. Variable, it seamlessly works.

Possible Solution

A workaround is to use tf.strided_slice instead of tf.slice. If we follow the source code of it, we will see, it takes the var argument which is a variable corresponding to input_

@tf_export("strided_slice")
@dispatch.add_dispatch_support
def strided_slice(input_,
                  begin,
                  end,
                  ..........
                  var=None,
                  name=None):

And when we pass a parameter for var that basically corresponding to the input_, it then calls assign function that is defined within it

def assign(val, name=None):
      """Closure that holds all the arguments to create an assignment."""

      if var is None:
        raise ValueError("Sliced assignment is only supported for variables")
      else:
        if name is None:
          name = parent_name + "_assign"

        return var._strided_slice_assign(
            begin=begin,
            end=end,
            strides=strides,
            value=val,
            name=name,
            begin_mask=begin_mask,
            end_mask=end_mask,
            ellipsis_mask=ellipsis_mask,
            new_axis_mask=new_axis_mask,
            shrink_axis_mask=shrink_axis_mask)

So, when we pass var in the tf.strided_slice, it will return an assignable object.

Code

Here is the full working code for reference.

import tensorflow as tf 
print(tf.__version__)

TYPE = tf.int32
N = 4
D = 5
shape = (N,D)

# Target to update
Y = tf.Variable(
    initial_value=tf.reshape(tf.range(N*D,dtype=TYPE), shape=shape),
    trainable=True
)
Y

2.4.1
<tf.Variable 'Variable:0' shape=(4, 5) dtype=int32, numpy=
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19]], dtype=int32)>

Now, we use tf.stried_slice instead of tf.slice.

S = tf.strided_slice(
    Y, 
    begin = [0, 1],
    end   = [3, 3],
    var   = Y,
    name  ='slice_op'
)
S

<tf.Tensor: shape=(3, 2), dtype=int32, numpy=
array([[ 1,  2],
       [ 6,  7],
       [11, 12]], dtype=int32)>

Update the variables with no attribution error.

# Values to set
V = tf.ones(shape=tf.shape(S), dtype=TYPE)
print(V)
print()

# Assing V to S region of T
S.assign(V)

tf.Tensor(
[[1 1]
 [1 1]
 [1 1]], shape=(3, 2), dtype=int32)

<tf.Variable 'UnreadVariable' shape=(4, 5) dtype=int32, numpy=
array([[ 0,  1,  1,  3,  4],
       [ 5,  1,  1,  8,  9],
       [10,  1,  1, 13, 14],
       [15, 16, 17, 18, 19]], dtype=int32)>

Using np like slicing.

# slicing 
S = Y[
    0:3,           
    1:3            
]                  
S
<tf.Tensor: shape=(3, 2), dtype=int32, numpy=
array([[ 1,  2],
       [ 6,  7],
       [11, 12]], dtype=int32)>

# Values to set
V = tf.ones(shape=tf.shape(S), dtype=TYPE)
print(V)
# Assing V to S region of T
S.assign(V)
tf.Tensor(
[[1 1]
 [1 1]
 [1 1]], shape=(3, 2), dtype=int32)
<tf.Variable 'UnreadVariable' shape=(4, 5) dtype=int32, numpy=
array([[ 0,  1,  1,  3,  4],
       [ 5,  1,  1,  8,  9],
       [10,  1,  1, 13, 14],
       [15, 16, 17, 18, 19]], dtype=int32)>

Materials

tf.Variable - tf.Tensor.