Tensorflow - ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float)

Question

Continuation from previous question: Tensorflow - TypeError: 'int' object is not iterable

My training data is a list of lists each comprised of 1000 floats. For example, x_train[0] =

[0.0, 0.0, 0.1, 0.25, 0.5, ...]

Here is my model:

model = Sequential()

model.add(LSTM(128, activation='relu',
               input_shape=(1000, 1), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(128, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1, activation='sigmoid'))

opt = tf.keras.optimizers.Adam(lr=1e-3, decay=1e-5)

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=3, validation_data=(x_test, y_test))

Here is the error I'm getting:

Traceback (most recent call last):
      File "C:\Users\bencu\Desktop\ProjectFiles\Code\Program.py", line 88, in FitModel
        model.fit(x_train, y_train, epochs=3, validation_data=(x_test, y_test))
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 728, in fit
        use_multiprocessing=use_multiprocessing)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 224, in fit
        distribution_strategy=strategy)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 547, in _process_training_inputs
        use_multiprocessing=use_multiprocessing)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 606, in _process_inputs
        use_multiprocessing=use_multiprocessing)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\data_adapter.py", line 479, in __init__
        batch_size=batch_size, shuffle=shuffle, **kwargs)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\data_adapter.py", line 321, in __init__
        dataset_ops.DatasetV2.from_tensors(inputs).repeat()
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 414, in from_tensors
        return TensorDataset(tensors)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 2335, in __init__
        element = structure.normalize_element(element)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\data\util\structure.py", line 111, in normalize_element
        ops.convert_to_tensor(t, name="component_%d" % i))
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1184, in convert_to_tensor
        return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1242, in convert_to_tensor_v2
        as_ref=False)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1296, in internal_convert_to_tensor
        ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\framework\tensor_conversion_registry.py", line 52, in _default_conversion_function
        return constant_op.constant(value, dtype, name=name)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 227, in constant
        allow_broadcast=True)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 235, in _constant_impl
        t = convert_to_eager_tensor(value, ctx, dtype)
      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 96, in convert_to_eager_tensor
        return ops.EagerTensor(value, ctx.device_name, dtype)
    ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float).

I've tried googling the error myself, I found something about using the tf.convert_to_tensor function. I tried passing my training and testing lists through this but the function won't take them.

Answer 1

TL;DR Several possible errors, most fixed with x = np.asarray(x).astype('float32').

Others may be faulty data preprocessing; ensure everything is properly formatted (categoricals, nans, strings, etc). Below shows what the model expects:

[print(i.shape, i.dtype) for i in model.inputs]
[print(o.shape, o.dtype) for o in model.outputs]
[print(l.name, l.input_shape, l.dtype) for l in model.layers]

---

The problem's rooted in using lists as inputs, as opposed to Numpy arrays; Keras/TF doesn't support former. A simple conversion is: x_array = np.asarray(x_list).

The next step's to ensure data is fed in expected format; for LSTM, that'd be a 3D tensor with dimensions (batch_size, timesteps, features) - or equivalently, (num_samples, timesteps, channels). Lastly, as a debug pro-tip, print ALL the shapes for your data. Code accomplishing all of the above, below:

Sequences = np.asarray(Sequences)
Targets   = np.asarray(Targets)
show_shapes()

Sequences = np.expand_dims(Sequences, -1)
Targets   = np.expand_dims(Targets, -1)
show_shapes()

# OUTPUTS
Expected: (num_samples, timesteps, channels)
Sequences: (200, 1000)
Targets:   (200,)

Expected: (num_samples, timesteps, channels)
Sequences: (200, 1000, 1)
Targets:   (200, 1)

---

As a bonus tip, I notice you're running via main(), so your IDE probably lacks a Jupyter-like cell-based execution; I strongly recommend the Spyder IDE. It's as simple as adding # In[], and pressing Ctrl + Enter below:

---

Function used:

def show_shapes(): # can make yours to take inputs; this'll use local variable values
    print("Expected: (num_samples, timesteps, channels)")
    print("Sequences: {}".format(Sequences.shape))
    print("Targets:   {}".format(Targets.shape))   

Answer 2

After trying everything above with no success, I found that my problem was that one of the columns from my data had boolean values. Converting everything into np.float32 solved the issue!

import numpy as np

X = np.asarray(X).astype(np.float32)

Answer 3

This should do the trick:

x_train = np.asarray(x_train).astype(np.float32)
y_train = np.asarray(y_train).astype(np.float32)

Answer 4

This is a HIGHLY misleading error, as this is basically a general error, which might have NOTHING to do with floats.

For example in my case it was caused by a string column of the pandas dataframe having some np.NaN values in it. Go figure!

Fixed it by replacing them with empty strings:

df.fillna(value='', inplace=True)

Or to be more specific doing this ONLY for the string (eg 'object') columns:

cols = df.select_dtypes(include=['object'])
for col in cols.columns.values:
    df[col] = df[col].fillna('')

Answer 5

Try with it for convert np.float32 to tf.float32 (datatype that read keras and tensorflow):

tf.convert_to_tensor(X_train, dtype=tf.float32)

Answer 6

I had many different inputs and target variables and didn't know which one was causing the problem.

To find out on which variable it breaks you can add a print value in the library package using the path is specified in your stack strace:

      File "C:\Users\bencu\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 96, in convert_to_eager_tensor
        return ops.EagerTensor(value, ctx.device_name, 

Adding a print statement in this part of the code allowed me to see which input was causing the problem:

constant_op.py:

  ....
      dtype = dtype.as_datatype_enum
    except AttributeError:
      dtype = dtypes.as_dtype(dtype).as_datatype_enum
  ctx.ensure_initialized()
  print(value) # <--------------------- PUT PRINT HERE
  return ops.EagerTensor(value, ctx.device_name, dtype)

After observing which value was problematic conversion from int to astype(np.float32) resolved the problem.

Answer 7

Could also happen due to a difference in versions (I had to move back from tensorflow 2.1.0 to 2.0.0.beta1 in order to solve this issue).

Answer 8

As answered by most people above, converting data in np.float32 / float32 by various means already told here.

While doing so if you get another error which is "ValueError: setting an array element with a sequence".

In that case, try converting your data into type list and then converting it into type tensor as you were trying earlier.

Answer 9

You'd better use this, it is because of the uncompatible version of keras

from keras import backend as K
X_train1 = K.cast_to_floatx(X_train)
y_train1 = K.cast_to_floatx(y_train)

Answer 10

You may want to check data types in input data set or array and than convert it to float32:

train_X[:2, :].view()
#array([[4.6, 3.1, 1.5, 0.2],
#       [5.9, 3.0, 5.1, 1.8]], dtype=object)
train_X = train_X.astype(np.float32)
#array([[4.6, 3.1, 1.5, 0.2],
#       [5.9, 3. , 5.1, 1.8]], dtype=float32)

Answer 11

Use this if you are using a DataFrame and has multiple columns type:

numeric_list = df.select_dtypes(include=[np.number]).columns
df[numeric_list] = df[numeric_list].astype(np.float32)

Answer 12

Just had the same issue and it ended up being because I was trying to pass an array of array objects, not an array of arrays as expected. Hope this helps someone in the future!

Answer 13

try

X_train =t ensorflow.convert_to_tensor(X_train, dtype=tensorflow.float32)
y_train = tensorflow.convert_to_tensor(y_train, dtype=tensorflow.float32)
X_test = tensorflow.convert_to_tensor(X_test, dtype=tensorflow.float32)
y_test = tensorflow.convert_to_tensor(y_test, dtype=tensorflow.float32)

Answer 14

The selected answer from OverLordGoldDragon provided valuable clues in my situation but I had to spend a few hours trying to rectify my particular situation. So adding some notes here to help any one else in a similar situation.

I am producing the input to my keras.Sequential model.fit() via the TimeseriesGenerator class from keras.preprocessing.sequences module. This class was outputting a 5d sequence with pythons inbuilt int/float types with following structure-

1. Level 5 - Batches(tuple of 10 batches since my batch size was 100 samples for a training set containing 1000 items)
2. Level 4 - X and Y variables( tuple of 2 n-dimensional numpy arrays, one representing the x variable and the other the y variable for the batch)
3. Level 3 3 dimensional nd numpy array of 100 samples each -for X variable a 100(batch size)x20(sequence length)x10(no of features) array, -for Y variable the last dimension is of size 1 instead of 10
4. Level 2 - each sample is a 2d array of 20(sequence length)x10(no of features) for x and 20x1 for y variable
5. Level 1 - is a scalar of native python types

When I followed the advise here and tried to convert this data structure to a x = np.asarray(x).astype('float32'), this was failing as the standard output from TimeseriesGenerator has tuples instead of arrays/lists at the first two levels which are immutable and in-homogenous in length (as the last batch has fewer samples than the others since it can only take in the residual number of samples (79 in my case)).

I tried a lot of things to make this casting work on the output of TimeseriesGEnerator, but finally I found the simplest solution was to explicitly cast the inputs to the TimeseriesGenerator constructor using the suggested method (x = np.asarray(x).astype('float32')). The problem was that these numpy nd arrays were being output by Preprocessing pipelines which retain native python int and float types in them. Once I cast these arrays to numpy float32 type, the model.fit() stopped giving the ValueError for failing to convert NumpyArray to Tensor Flow.

Hoping this note helps someone else who is facing this issue.

Answer 15

In my case, it didn't work to cast to np.float32.

For me, everything ran normally during training (probably because I was using tf.data.Dataset.from_generator as input for fit()), but when I was trying to call predict() on 1 instance (using a np.array), the error shows up.

As a solution, I had to reshape the array x_array.reshape(1, -1) before calling predict and it worked.

Answer 16

I avoided this problem by enforcing floating-point format during data import:

df = pd.read_csv('titanic.csv', dtype='float')

Answer 17

I encountered the same issue with some Pandas Series, thinking

data = my_series.to_numpy()

would be enough, but it would only provide an Object dtype and forcing to float64 or whatever didn’t work.

The issue was solved by using

data = my_series.to_list()

instead. Then

dataset = tf.data.Dataset.from_tensor_slices((data, labels))

worked as intended.