Does Tensorflow have optimizers for compute graphs involving complex64 tensors?

Question

The tf.float32 version of the following code works. However, when we try to run the following, we get an exception (on the line where we try to define an optimizer).

import tensorflow as tf
tf.InteractiveSession()

complex_weights = tf.Variable(tf.complex(tf.truncated_normal([3, 4]), 
                                         tf.truncated_normal([3, 4])))

complex_target = tf.complex(tf.zeros([3,4]), 
                            tf.zeros([3,4]))
loss = tf.reduce_mean(tf.pow(tf.real(complex_weights - complex_target), 2))
optimizer = tf.train.AdamOptimizer(1.0).minimize(loss)

Here is the error message from the top of the stack:

....../local/lib/python2.7/site-packages/tensorflow/python/training/optimizer.pyc in _assert_valid_dtypes(self, tensors)
    352         raise ValueError(
    353             "Invalid type %r for %s, expected: %s." % (
--> 354                 dtype, t.name, [v for v in valid_dtypes]))
    355 
    356   # --------------

ValueError: Invalid type tf.complex64 for Variable_3:0, expected: [tf.float32].

Does this mean that complex numbers are not supported as intermediate steps when trying to do optimization? Short of manually encoding complex number operations, are there any workarounds?

Answer 1

Yes, you hit intermediate op that didn't have complex support. See this question for some pointers to code where you can see which data types are registered for which devices. In general I think complex support is fairly slim, most ops don't support it.