7

I need to change the value of some elements of a tensor. I know what elements -- they are in a boolean tensor already.

I don't see how to do this in keras code. But if I were using TensorFlow code I would do something like this:

Conditional assignment of tensor values in TensorFlow

In python numpy, the code would look something like this: 

x = np.zeros_like(sometensor) 
x[sometensor>0.5] = 1.0

In Keras code (and I'm using TF backend) here's my best attempt (does not work):

encoder_outputs_bin = k.backend.zeros_like(encoder_outputs, name="encoder_outputs_bin")
point_five = k.backend.constant(0.5, shape=k.backend.shape(encoder_outputs), name="point_five")
positives = k.backend.greater_equal(encoder_outputs, point_five)
encoder_outputs_bin[positives].assign(tf.ones(1)) # TF syntax -- might not work in keras
desertnaut
  • 57,590
  • 26
  • 140
  • 166
Geoffrey Anderson
  • 1,534
  • 17
  • 25
  • Related: https://stackoverflow.com/questions/48795910/binarize-tensor-in-keras – Geoffrey Anderson Jul 23 '18 at 19:40
  • Following code seems nice but Keras throws error at train time: encoder_outputs = Dense(units=latent_vector_len, activation='sigmoid', kernel_initializer="lecun_normal")(x) encoder_outputs_bin = k.layers.Lambda(lambda z: k.backend.round(z))(encoder_outputs) – Geoffrey Anderson Jul 24 '18 at 22:27
  • Is there a trivially easy solution using categorical cross entropy loss, which somehow performs the binarization internally and computes loss on that 2-value categorical variable? – Geoffrey Anderson Jul 24 '18 at 22:29
  • Round() is not differentiable, causing errors in Keras when Keras tries to compute gradient it seems. However Relu is not differentiable but it works fine as everyone knows, so maybe there is similarly a way to get Round to make Keras happy. – Geoffrey Anderson Jul 24 '18 at 22:34
  • 1
    Relu is differentiable. Only when it's in the zero region it gets a zero gradient. (But the movement of the other vars which are in the differentiable zone may rescue the others from the zero or not). --- Relu is ok as long as you don't have the unfortunate case of all elements going to 0, then your model gets stuck). – Daniel Möller Jul 25 '18 at 00:32
  • As an easy solution for many possible classes, you can use `binary_crossentropy`. (Compared to `categorical_crossentropy`, binary can take many true classes, while categorical takes only one). – Daniel Möller Jul 25 '18 at 00:33

2 Answers2

0

This answer is not really "assign", it's getting another tensor, but I believe it will do...

Also, what you intend to do will totally break backpropagation for these elements.

Knowing this:

positives = k.backend.greater_equal(encoder_outputs, 0.5)
positives = k.backend.cast(positives, k.backend.floatx())

encoder_outputs = positives + ((1-positives)*encoder_outputs)
Daniel Möller
  • 84,878
  • 18
  • 192
  • 214
0

A lambda function in the activation worked for me. It is one step more complicated than the simple use of a built-in activation function.

encoder_outputs = Dense(units=latent_vector_len, activation=k.layers.Lambda(lambda z: k.backend.round(k.layers.activations.sigmoid(x=z))), kernel_initializer="lecun_normal")(x)

This code changes the output of a Dense from Reals to 0,1 (ie, binary).

Keras throws a warning but the code still proves to work.

# Look it works!

y = encoder_model.predict(x=x_in)
print(y)

>>>[[1. 0. 0. 1. 0. 1. 0. 0.]]
Geoffrey Anderson
  • 1,534
  • 17
  • 25
  • Round is one of the functions that will not backpropagate, though. :( When fitting, you will see either "An operation has None for gradient" or "None values not supported". – Daniel Möller Jul 25 '18 at 00:16