5

I am trying to use Sparse numpy matrix for keras with tensorflow as backend. The model compiles but while fit, gives an error. Code is as follows. Any help is appreciated.

from keras.layers import Dense, Input
from keras.models import Model
inputs = Input(shape=(trainX.shape[1],), sparse=True)
outputs = Dense(trainY.shape[1], activation='softmax')(inputs)
model = Model(inputs=inputs, outputs=outputs)
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

trainX is 

<2404941x337071 sparse matrix of type '<type 'numpy.float64'>'
with 4765705 stored elements in Compressed Sparse Row format>

and similarly trainY is a CSR matrix

model.fit(trainX, trainY, verbose=1)

gives following error

ValueError: setting an array element with a sequence.
TheRajVJain
  • 390
  • 5
  • 15
  • 1
    If the documentation (e.g. for `fit`) doesn't say an argument can be a sparse matrix, then it can't. https://stackoverflow.com/questions/36803689/does-keras-1-0-support-scipy-sparse-matrices. Often, if an input is not an array, code will wrap it in `np.asarray(trainX)`. If the input is sparse, the result is a 1 element object array, which is not what you want. – hpaulj Oct 05 '17 at 06:49
  • Yeah, that's what I thought. But why would they have a sparse option then? and here https://github.com/fchollet/keras/issues/4984 some guys claim to have solved the problem @hpaulj – TheRajVJain Oct 05 '17 at 06:57
  • And here too: https://rstudio-pubs-static.s3.amazonaws.com/260770_e9f31a39cea94ff1b5616fe8b6b4ff28.html#18 – TheRajVJain Oct 05 '17 at 07:00
  • 1
    I think that for sparse tensors, a custom batch generator is used that converts just a part of the tensor back into its dense form. Ref: https://stackoverflow.com/questions/37609892/keras-sparse-matrix-issue – Sussch Nov 28 '18 at 09:11

1 Answers1

2

It is possible to use sparse matrices as inputs to a Keras model if you write a custom training loop. In the example below, the model takes a sparse matrix as an input and outputs a dense matrix.

from keras.layers import Dense, Input
from keras.models import Model
import scipy
import numpy as np

trainX = scipy.sparse.random(1024, 1024)
trainY = np.random.rand(1024, 1024)

inputs = Input(shape=(trainX.shape[1],), sparse=True)
outputs = Dense(trainY.shape[1], activation='softmax')(inputs)
model = Model(inputs=inputs, outputs=outputs)
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

steps = 10
for i in range(steps):
  # For simplicity, we directly use trainX and trainY in this example
  # Usually, this is where batches are prepared
  print(model.train_on_batch(trainX, trainY))
# [3549.2546, 0.0]
# ...
# [3545.6448, 0.0009765625]

From your example, it seems that you would like your output to be a sparse matrix too. This is more difficult as your model needs to output a sparse matrix and your loss has to be computable with sparse matrices. Moreover, I believe Keras does not support sparse outputs yet.

Maurice Qch
  • 166
  • 1
  • 4
  • I got this error -> Attempt to convert a value (<1024x1024 sparse matrix of type '' with 10485 stored elements in COOrdinate format>) with an unsupported type () to a Tensor. – Areza Apr 15 '20 at 19:06