which value is used for padding in tensorflow convolution?

Question

In PyTorch, there are two choices for padding_mode when using convolution: zeros and circular.

However, I cannot find a similar parameter in the docs of TensorFlow.

In addition, I find that TensorFlow is not using 0 as the value to pad the inputs.

Does anyone know the answer?

Answer 1

It does use zeros:

import tensorflow as tf

inputs = tf.ones((1, 10, 1))

Imagine strides of 2 with a filter of 3 (made only of 1s)

<tf.Tensor: shape=(1, 10, 1), dtype=float32, numpy=
array([[[1.],  |
        [1.],  | 3  
        [1.],  |   |          
        [1.],      | 3      
        [1.],      |    |  
        [1.],           | 3  
        [1.],           |    | 
        [1.],                | 3           
        [1.],                |    |
        [1.]                      | 2
        # (will add a zero here)
]], dtype=float32)>

All the elements from the operation will result in sum(1, 1, 1). If the last one is padded with zeros, the result will be sum(1, 1, 0), which it is:

conv = tf.keras.layers.Conv1D(filters=1,
                              kernel_size=3,
                              strides=2,
                              padding='SAME',
                              kernel_initializer=tf.keras.initializers.Ones)

tf.squeeze(conv(inputs))

<tf.Tensor: shape=(5,), dtype=float32, numpy=array([3., 3., 3., 3., 2.], dtype=float32)>

You can also see/demonstrate this using tf.nn.conv1d:

import tensorflow as tf

inputs = tf.ones((1, 10, 1))

result = tf.nn.conv1d(input=inputs,
                      filters=tf.ones((3, 1, 1)),
                      stride=2,
                      padding='SAME')

tf.squeeze(result)

<tf.Tensor: shape=(5,), dtype=float32, numpy=array([3., 3., 3., 3., 2.], dtype=float32)>