-1

I did the following simple calculation in TensorFlow and was surprised to see that the result is not as I expected. My code:

a = tf.constant([[0.2,1],[0.3,0.75]])
b = tf.constant([[0.2,0.1],[0.8,0.2]])
print(tf.square(a)+tf.square(b))

It delivers the result:

tf.Tensor(
[[0.08000001 1.01      ]
 [0.73       0.6025    ]], shape=(2, 2), dtype=float32)

However, I would have expected:

tf.Tensor(
[[0.08       1.0     ]
 [0.73       0.6025    ]], shape=(2, 2), dtype=float32)

since this would be the mathematically correct result of my calculation. What went wrong?

Katja
  • 11
  • 1
  • 5

1 Answers1

1

When using float32 accuracy, the result

>>> tf.square(0.2).numpy()
0.040000003

meaning that

>>> (tf.square(a)+tf.square(b))[0][0].numpy()
0.080000006

When the whole resulting array is printed, the element is rounded to 0.08000001.

Also,

>>> (tf.square(1.0)+tf.square(0.1)).numpy()
1.01

as it should be, meaning TensorFlow is doing the calculations correctly.

sakumoil
  • 602
  • 4
  • 11
  • Well, (0.2)^2 = 0.04 so TensorFlow is giving the wrong result with 0.040000003. So that does not really answer my question. Do you mean that the wrong result in TensorFlow occurs due to insufficient bit length of float32? What is the common practice to avoid this problem? – Katja Jul 14 '21 at 13:18
  • Seems like using `tf.float16` datatype yields the correct results: `>>> tf.cast(tf.square(0.2), tf.float16).numpy() 0.04` – sakumoil Jul 15 '21 at 05:19
  • Can you please explain the underlying concept? I need this to work for every example, not just for this one. It does not make sense to me that a smaller bitlength of float would yield better results - I assume that this only works better for this specific example. – Katja Jul 16 '21 at 08:40