I have a py_func in my dataset.map callback that returns a sample weight. See this example:
def get_sample_weight(team, number):
# Some code
return np.array([0.5]).astype(np.float32) #return weight
def preprocess_tfrecord_example(example, sample_weights):
image = tf.image.decode_png(example['detection/encoded'])
team = example['detection/team_name']
sample_weight = tf.py_func(get_sample_weight, [team, example['detection/number']], tf.float32)
image = tf.image.resize_image_with_pad(image,56,108)
number = tf.expand_dims(tf.cast(example['detection/number'], tf.float32), -1)
return image/255., number, sample_weight
However, when I create the dataset and call model.fit(dataset, ....) this gives
ValueError: Cannot take the length of Shape with unknown rank.
This is because Tensorflow cannot determine the shape returned by the python function, see Output from TensorFlow `py_func` has unknown rank/shape.
As a workaround I try to set the shape with sample_weight.set_shape(1) before returning it in preprocess_tfrecord_example(..).
But this gives
ValueError: Found a sample_weight array with shape (?, 1). In order to use timestep-wise sample weights, you should specify sample_weight_mode="temporal" in compile(). If you just mean to use sample-wise weights, make sure your sample_weight array is 1D.
How can I set the right shape, such that the sample_weight array for use in model.fit is 1D?
