Tensor construction with a loop over number of batches

Question

I want to create a tensor which is some kind of a transformation matrix (rotation matrix for instance)

My model predicts 2 parameters: x1 and x2

so the output is a tensor of (B, 2), when B is number of batches.

however, when I write my loss, I have to know this "B" since I want to iterate over it:

def get_rotation_tensor(x):
    roll_mat = K.stack([ [[1, 0, 0],
                          [0, K.cos(x[i, 0]), -K.sin(x[i, 0])],
                          [0, K.sin(x[i, 0]), K.cos(x[i, 0])]] for i in range(BATCH_SIZE)])
    pitch_mat = K.stack([ [[K.cos(x[i, 1]), 0, K.sin(x[i, 1])],
                           [0, 1, 0],
                           [-K.sin(x[i, 1]), 0, K.cos(x[i, 1])]] for i in range(BATCH_SIZE)])
    return K.batch_dot(pitch_mat, roll_mat)

the only solution I could have think of is to pre-define the BATCH_SIZE in advance.. but is there a way to write a general loss function that will work for every batch size?

THANKS

score 0 · Answer 1 · answered Nov 15 '19 at 13:10

I found a solution

def get_rotation_tensor(x):
    ones = K.ones_like(x[:, 0])
    zeros = K.zeros_like(x[:, 0])
    roll_mat = K.stack([[ones, zeros, zeros],
                          [zeros, K.cos(x[:, 0]), -K.sin(x[:, 0])],
                          [zeros, K.sin(x[:, 0]), K.cos(x[:, 0])]])
    pitch_mat = K.stack([[K.cos(x[:, 1]), zeros, K.sin(x[:, 1])],
                           [zeros, ones, zeros],
                           [-K.sin(x[:, 1]), zeros, K.cos(x[:, 1])]])
    return K.batch_dot(K.permute_dimensions(pitch_mat, (2, 0, 1)), 
                       K.permute_dimensions(roll_mat, (2, 0, 1)))

I discovered something that might be useful. If you decorate a function with @tf.function, you are able to take certain Python constructs and Tensorflow 2.0 will convert them into a graph. This might help in certain cases so I wanted to share. Additional details here: https://www.tensorflow.org/guide/function I hope this helps. — ad2004, Dec 01 '19 at 19:12

score -1 · Answer 2 · answered Nov 13 '19 at 17:05

Perhaps I'm not fully understanding your issue, but can't you just determine the batch size by the shape of the tensors passed into the loss function. Below is an example that shows the idea. I hope this helps.

# Install TensorFlow
try:
  # %tensorflow_version only exists in Colab.
  %tensorflow_version 2.x
except Exception:
  pass

import tensorflow as tf
print(tf.__version__)
print(tf.executing_eagerly())

# Setup repro section from Keras FAQ with TF1 to TF2 adjustments

import numpy as np
import random as rn

# The below is necessary for starting Numpy generated random numbers
# in a well-defined initial state.

np.random.seed(42)

# The below is necessary for starting core Python generated random numbers
# in a well-defined state.

rn.seed(12345)

# Force TensorFlow to use single thread.
# Multiple threads are a potential source of non-reproducible results.
# For further details, see: https://stackoverflow.com/questions/42022950/

session_conf = tf.compat.v1.ConfigProto(intra_op_parallelism_threads=1,
                                        inter_op_parallelism_threads=1)

# The below tf.set_random_seed() will make random number generation
# in the TensorFlow backend have a well-defined initial state.
# For further details, see:
# https://www.tensorflow.org/api_docs/python/tf/set_random_seed

tf.compat.v1.set_random_seed(1234)

sess = tf.compat.v1.Session(graph=tf.compat.v1.get_default_graph(), config=session_conf)
tf.compat.v1.keras.backend.set_session(sess)

# Rest of code follows ...

# Custom Loss
def my_custom_loss(y_true, y_pred):

    tf.print('inside my_custom_loss:')
    tf.print('y_true:')
    tf.print(y_true)
    tf.print('y_true column 0:')
    tf.print(y_true[:,0])
    tf.print('y_true column 1:')
    tf.print(y_true[:,1])
    tf.print('y_pred:')
    tf.print(y_pred)

# get length/batch size

    batch_size=tf.shape(y_pred)[0]
    tf.print('batch_size:')
    tf.print(batch_size)

    y_zeros = tf.zeros_like(y_pred)
    y_mask = tf.math.greater(y_pred, y_zeros)
    res = tf.boolean_mask(y_pred, y_mask)
    logres = tf.math.log(res)
    finres = tf.math.reduce_sum(logres)

    return finres

# Define model
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(1, activation='linear', input_dim=1, name="Dense1"))
model.compile(optimizer='rmsprop', loss=my_custom_loss)
print('model.summary():')
print(model.summary())

# Generate dummy data
data = np.array([[2.0],[1.0],[1.0],[3.0],[4.0]])
labels = np.array([[[2.0],[1.0]],
                   [[0.0],[3.0]],
                   [[0.0],[3.0]],
                   [[0.0],[3.0]],
                   [[0.0],[3.0]]])

# Train the model.
print('training the model:')
print('-----')
model.fit(data, labels, epochs=1, batch_size=3)
print('done training the model.')

print(data.shape)
print(labels.shape)

No this its shape is None when compiling the model. When u do 'int_shape' it returns None, the u can't run a loop 'in range(none)' — Jenia Golbstein, Nov 13 '19 at 19:10

Tensor construction with a loop over number of batches

2 Answers2