Trying to build a custom keras layer for GLCM, which requires to count pairs of pixels before doing further calculations, gets me the error below. The algorithm works just fine outside keras context.
Error 1
Only integers, slices (`:`), ellipsis (`...`), tf.newaxis (`None`) and scalar tf.int32/tf.int64 tensors are valid indices, got <tf.Tensor 'glcm_layer_impl/strided_slice:0' shape=(150, 1) dtype=float32>
I understand, that i,j at the end of the loop are not integers with pixels values in graph mode, but SliceTensors. So I tried casting it to int(), but just to realize I get a CastTensor object instead.
Layer definition:
class GLCMLayer(keras.layers.Layer):
def __init__(self, greylevels=32, angle=0, name=None, **kwargs):
self.greylevels = greylevels
self.angle = angle
super(GLCMLayer, self).__init__(name=name, **kwargs)
def _glcm_loop(self, image, angle, levels, out):
rows = image.shape[1] # batch dimension is first entry
cols = image.shape[2]
offset_row = round(sin(angle))
offset_col = round(cos(angle))
start_row = max(0, -offset_row)
end_row = min(rows, rows - offset_row)
start_col = max(0, -offset_col)
end_col = min(cols, cols - offset_col)
for r in range(start_row, end_row):
for c in range(start_col, end_col):
i = image[r, c]
row = r + offset_row
col = c + offset_col
j = image[row, col]
out[i, j, 0] += 1 # here the errors occur
def call(self, inputs):
P = tf.experimental.numpy.zeros((self.greylevels, self.greylevels, 1), dtype=np.uint32) # using np.zeros() will yield a different error
self._glcm_loop(inputs, self.angle, self.greylevels, P)
return P
Trying different way likes eval() and this, didnt't help me either.
Then I tried out[0,0,0]+=1 just to see if that works, but didn't:
Error 2
'Tensor' object does not support item assignment
Now I learned that Tensors are immutable. I tried other ways to get around it: using tf.Variables or using plain python object like this. Yet I cannot think of a way on how to create a list and increase a counter for each pair of found pixels, because or Error 1.
Either way it always ended up with the first or second error. This seems to be the best idea, to get around it. Coming mostly from OOP programming, I cannot think of all the tf-functions I need here. I will try to get to know these functions, but maybe somebody already has an idea to get along without the tf-functions.
Greatly appreciate any help on how to resolve above errors. Let me know if there is more than I can add to this question.
Input images are grayscale with 32 values and batched.
Model definition:
inputs = keras.Input(shape=input_shape)
model = GLCMLayerImpl()(inputs)
model = layers.Flatten()(model)
model = layers.Dense(64, activation="relu")(model)
outputs = layers.Dense(num_classes, activation="softmax")(model)
