ValueError: Tried to convert 'y' to a tensor and failed. Error: None values not supported

Question

DOESN'T WORK:

from tensorflow.python.keras.layers import Input, Dense
from tensorflow.python.keras.models import Model
from tensorflow.python.keras.optimizers import Nadam
import numpy as np

ipt = Input(shape=(4,))
out = Dense(1, activation='sigmoid')(ipt)

model = Model(ipt, out)
model.compile(optimizer=Nadam(lr=1e-4), loss='binary_crossentropy')

X = np.random.randn(32,4)
Y = np.random.randint(0,2,(32,1))
model.train_on_batch(X,Y)

WORKS: remove .python from above's imports.

What's the deal, and how to fix?

---

ADDITIONAL INFO:

• CUDA 10.0.130, cuDNN 7.4.2, Python 3.7.4, Windows 10
• tensorflow, tensorflow-gpu v2.0.0, and Keras 2.3.0 via pip, all else via Anaconda 3
• Per DEBUG 1, I note pip installs the r2.0 branch rather than master; manually overwriting local tensorflow_core.python folder with master's breaks everything - but doing so for a select-few files doesn't, though error persists

---

DEBUG 1: files difference

This holds for my local installation, rather than TF's Github branches master or r2.0; Github files lack api/_v2 for some reason:

from tensorflow import keras
print(keras.__file__)
from tensorflow.python import keras
print(keras.__file__)

[1] D:\Anaconda\envs\tf2_env\lib\site-packages\tensorflow_core\python\keras\api\_v2\keras\__init__.py
[2] D:\Anaconda\envs\tf2_env\lib\site-packages\tensorflow_core\python\keras\__init__.py

Looking into each __init__ for Optimizer:

# [1]
from tensorflow.python.keras.optimizer_v2.optimizer_v2 import OptimizerV2 as Optimizer
# [2]
from tensorflow.python.keras import optimizers

# in python.keras.optimizers.py:
# all imports are from tensorflow.python
class Optimizer(object): # <--- does NOT use optimizer_v2 for Optimizer

This appears to root the problem, as below works:

from tensorflow.python.keras.layers import Input, Dense
from tensorflow.python.keras.models import Model
from tensorflow.keras.optimizers import Nadam

This is strange, however, as the direct import keras doesn't use optimizer_v2 either, though the definition of Optimizer in keras.optimizers does differ.

---

DEBUG 2: execution difference

Debugging side-by-side, while both use the same training.py, execution diverges fairly quickly:

### TF.KERAS
    if self._experimental_run_tf_function: #  TRUE
### TF.PYTHON.KERAS
    if self._experimental_run_tf_function: #  FALSE

Former proceeds to call training_v2_utils.train_on_batch(...) and returns thereafter, latter self._standardize_user_data(...) and others before ultimately failing.

---

DEBUG 3 (+ solution?): the fail-line

if None in grads: # <-- in traceback

Inserting print(None in grads) right above it yields the exact same error - thus, it appears related to TF2 iterable ops -- this works:

if any([g is None for g in grads]): # <-- works; similar but not equivalent Python logic

Unsure yet if it's a complete fix, still debugging -- update: started a Github Pull Request

---

Full error trace:

  File "<ipython-input-1-2db039c052cf>", line 20, in <module>
    model.train_on_batch(X,Y)
  File "D:\Anaconda\envs\tf2_env\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 1017, in train_on_batch
    self._make_train_function()
  File "D:\Anaconda\envs\tf2_env\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 2116, in _make_train_function
    params=self._collected_trainable_weights, loss=self.total_loss)
  File "D:\Anaconda\envs\tf2_env\lib\site-packages\tensorflow_core\python\keras\optimizers.py", line 653, in get_updates
    grads = self.get_gradients(loss, params)
  File "D:\Anaconda\envs\tf2_env\lib\site-packages\tensorflow_core\python\keras\optimizers.py", line 92, in get_gradients
    if None in grads:
  File "D:\Anaconda\envs\tf2_env\lib\site-packages\tensorflow_core\python\ops\math_ops.py", line 1336, in tensor_equals
    return gen_math_ops.equal(self, other)
  File "D:\Anaconda\envs\tf2_env\lib\site-packages\tensorflow_core\python\ops\gen_math_ops.py", line 3626, in equal
    name=name)
  File "D:\Anaconda\envs\tf2_env\lib\site-packages\tensorflow_core\python\framework\op_def_library.py", line 545, in _apply_op_helper
    (input_name, err))

ValueError: Tried to convert 'y' to a tensor and failed. Error: None values not supported.

Answer 1

It was a bug, and my pull request fix was approved (but isn't yet merged). In the meantime, you can make the change manually, as here. Also, tf.python.keras isn't always meant to be used, if at all.

UPDATE: the pull request is now merged.

---

Why it works: None in grads is same as any(g == None for g in grads); problem is, g may be a tf.Tensor/tf.Variable which has .__eq__ defined only to operate on tensors, so is None must be used instead.

from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model
import numpy as np

ipt = Input((16,))
out = Dense(16)(ipt)
model = Model(ipt, out)
model.compile('adam', 'mse')

x = y = np.random.randn(32, 16)
model.train_on_batch(x, y)

W = model.optimizer.weights
W[0] == None

>>> ValueError: Attempt to convert a value (None) with an unsupported type 
    (<class 'NoneType'>) to a Tensor.

Checking source code:

from inspect import getsource
print(getsource(W[0].__eq__))

def __eq__(self, other):
    """Compares two variables element-wise for equality."""
    if ops.Tensor._USE_EQUALITY and ops.executing_eagerly_outside_functions():
        return gen_math_ops.equal(self, other, incompatible_shape_error=False)
    else:
        # In legacy graph mode, tensor equality is object equality
        return self is other

Answer 2

Probably you should correct your imports

from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Nadam