1

EDIT: I got past that error message by reshaping my data as follows:

    train_x = np.array(train_x)
    train_y = np.array(train_y)
    x_size = train_x.shape[0] * train_x.shape[1]
    train_x = train_x.reshape(x_size, train_x.shape[2])
    train_x = np.expand_dims(train_x, 1)
    train_x = train_x.transpose(0,2,1)
    train_y = train_y.flatten()
    shape = train_x.shape # 3D: number of texts * number of padded paragraphs, number of features, 1
    time_steps = shape[0]  # number of padded pars * number of texts
    features = shape[1]  # number of features

    model = Sequential()
    model.add(layers.Masking(mask_value=0, input_shape=(time_steps, features)))
    model.add(layers.LSTM(128, return_sequences=True, return_state=False, input_shape=(time_steps, features)))  # 128 internal units
    model.add(layers.TimeDistributed(layers.Dense(1, activation='sigmoid')))
    #model.add(layers.Dense(len(train_y)))  # Dense layer
    model.compile(loss='binary_crossentropy', optimizer='adam')

    model.fit(train_x, train_y, batch_size=train_y.shape[0])

    predictions = model.predict(test_x)

I get a new error message:

ValueError: Input 0 is incompatible with layer lstm: expected shape=(None, None, 3), found shape=[288, 3, 1]

I'll keep updating this question in case someone runs into a similiar problem. Still happy about any input.

Original question: I want to buil a sequential LSTM model that predicts binary classification at every time step. More exactly, I want to predict an output for every paragraph in my texts (48 is the number of paragraphs). This is my code:

shape = np.shape(train_x) # 3D: number of texts, number of padded paragraphs, number of features
n = shape[0]  # number of texts
time_steps = shape[1]  # number of padded pars
features = shape[2]  # number of features

model = Sequential()
model.add(layers.Masking(mask_value=0.0, input_shape=(time_steps, features)))
model.add(layers.LSTM(128, return_sequences=True, return_state=False))  
model.add(layers.TimeDistributed(layers.Dense(1)))
model.compile(loss='categorical_crossentropy', optimizer='adam')
model.summary()

#train_x = np.array(train_x).reshape(2, input_shape, 3)
train_x = tf.convert_to_tensor(train_x)  # data needs to be tensor object
train_y = tf.convert_to_tensor(train_y)
model.fit(train_x, train_y, batch_size=2)

predictions = model.predict(test_x)

This is the error message I get:

ValueError: Can not squeeze dim[1], expected a dimension of 1, 
got 48 for '{{node categorical_crossentropy/weighted_loss/Squeeze}} = Squeeze[T=DT_FLOAT, 
squeeze_dims=[-1]](Cast)' with input shapes: [2,48].

I don't really know what to do with this, do I need to reshape my data? How? Or do I need to change something in the model? Thanks! (changing the loss function to 'binary_crossentropy' raises the same error)

This is the entire traceback:

Traceback (most recent call last):
  File "program.py", line 247, in <module>
    eval_scores = train_classifier(x_train, y_train_sc, x_test, y_test_sc)
  File "program.py", line 201, in train_classifier
    model.fit(train_x, train_y, batch_size=2)
  File "C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py", line 108, in _method_wrapper
    return method(self, *args, **kwargs)
  File "C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1098, in fit
    tmp_logs = train_function(iterator)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\def_function.py", line 780, in __call__
    result = self._call(*args, **kwds)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\def_function.py", line 823, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\def_function.py", line 696, in _initialize
    self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\function.py", line 2855, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\function.py", line 3213, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\function.py", line 3065, in _create_graph_function
    func_graph_module.func_graph_from_py_func(
  File "C:\Python38\lib\site-packages\tensorflow\python\framework\func_graph.py", line 986, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "C:\Python38\lib\site-packages\tensorflow\python\eager\def_function.py", line 600, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "C:\Python38\lib\site-packages\tensorflow\python\framework\func_graph.py", line 973, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

    C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py:806 train_function  *
        return step_function(self, iterator)
    C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py:796 step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    C:\Python38\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:1211 run
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    C:\Python38\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2585 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    C:\Python38\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2945 _call_for_each_replica
        return fn(*args, **kwargs)
    C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py:789 run_step  **
        outputs = model.train_step(data)
    C:\Python38\lib\site-packages\tensorflow\python\keras\engine\training.py:748 train_step
        loss = self.compiled_loss(
    C:\Python38\lib\site-packages\tensorflow\python\keras\engine\compile_utils.py:204 __call__
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    C:\Python38\lib\site-packages\tensorflow\python\keras\losses.py:150 __call__
        return losses_utils.compute_weighted_loss(
    C:\Python38\lib\site-packages\tensorflow\python\keras\utils\losses_utils.py:111 compute_weighted_loss
        weighted_losses = tf_losses_utils.scale_losses_by_sample_weight(
    C:\Python38\lib\site-packages\tensorflow\python\ops\losses\util.py:142 scale_losses_by_sample_weight
        losses, _, sample_weight = squeeze_or_expand_dimensions(
    C:\Python38\lib\site-packages\tensorflow\python\ops\losses\util.py:95 squeeze_or_expand_dimensions
        sample_weight = array_ops.squeeze(sample_weight, [-1])
    C:\Python38\lib\site-packages\tensorflow\python\util\dispatch.py:201 wrapper
        return target(*args, **kwargs)
    C:\Python38\lib\site-packages\tensorflow\python\util\deprecation.py:507 new_func
        return func(*args, **kwargs)
    C:\Python38\lib\site-packages\tensorflow\python\ops\array_ops.py:4259 squeeze
        return gen_array_ops.squeeze(input, axis, name)
    C:\Python38\lib\site-packages\tensorflow\python\ops\gen_array_ops.py:10043 squeeze
        _, _, _op, _outputs = _op_def_library._apply_op_helper(
    C:\Python38\lib\site-packages\tensorflow\python\framework\op_def_library.py:742 _apply_op_helper
        op = g._create_op_internal(op_type_name, inputs, dtypes=None,
    C:\Python38\lib\site-packages\tensorflow\python\framework\func_graph.py:591 _create_op_internal
        return super(FuncGraph, self)._create_op_internal(  # pylint: disable=protected-access
    C:\Python38\lib\site-packages\tensorflow\python\framework\ops.py:3477 _create_op_internal
        ret = Operation(
    C:\Python38\lib\site-packages\tensorflow\python\framework\ops.py:1974 __init__
        self._c_op = _create_c_op(self._graph, node_def, inputs,
    C:\Python38\lib\site-packages\tensorflow\python\framework\ops.py:1815 _create_c_op
        raise ValueError(str(e))

    ValueError: Can not squeeze dim[1], expected a dimension of 1, got 48 for '{{node categorical_crossentropy/weighted_loss/Squeeze}} = Squeeze[T=DT_FLOAT, squeeze_dims=[-1]](Cast)' with input shapes: [2,48].
maeven
  • 21
  • 4
  • As your codes doesn't seem to be invoking any `squeeze` function, the only possibility is that this error is happening deep inside some external code that has been called from your code. The next question is where is exactly did your code call the external code that threw this error. The only way to answer that is by looking at the complete traceback of the error, especially the upper parts of the traceback. The part of the error that you've posted seems to be only a part of the traceback. – fountainhead Nov 21 '20 at 12:50
  • Thanks for the help! I don't really understand the entire traceback, I added it in the question now. I tried to vary the batch size and it seems to vary the error message, but doesn't fix it. – maeven Nov 21 '20 at 14:49

0 Answers0