0

I know this kind of problem has been already answered ( here or here), but all the answers didn't work for me:

My X is a vector of size 5000.

Each xi is a sparse matrix of size (190000,42) And the y is a vector of size 5000 and each yi is a vector of size 190000 (sparse)

X =
array([ <191483x42 sparse matrix of type '<class 'numpy.float64'>'
      with 75431 stored elements in Compressed Sparse Row format>,
      <191483x42 sparse matrix of type '<class 'numpy.float64'>'
      with 182015 stored elements in Compressed Sparse Row format>,
      <191483x42 sparse matrix of type '<class 'numpy.float64'>',], dtype=object)

I want to make a very simple model for the moment, using TimeDistributed. (This work fine when I have only 100 matrix in my vector X.)

I try to use fit_generator to unsparse each matrix but this didn't work.

def build_model(): 
   model = Sequential()

   model.add(TimeDistributed(Dense(1, bias=0, W_regularizer=regularizers.l1(0.01)), input_shape=(191483, 42)))
   model.add(Activation("softmax"))
   model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])
   return model

print ('load model')
model = build_model()


def batch_generator(X, y):
    for i in range(len(X)):
        X_array = X[i].toarray()
        y_array = y[i].toarray()
        yield (X_array,y_array)

model.fit_generator(generator=batch_generator(X, y), samples_per_epoch=10000, nb_epoch=10)

I got this error: 

ValueError: Error when checking model input: expected timedistributed_input_1 to have 3 dimensions, but got array with shape (191483, 42)

Thx for your help

Williamben
  • 191
  • 2
  • 7
  • can you clarify on how the other answers didnt work for you? – WhatsThePoint Jul 11 '17 at 07:45
  • because the issue of the previous question was to deal with one big sparse matrix. My problem is different, because each matrix is not very big, but the vector of 5000 is very big and i have to convert each matrix to a sparse matrix to store it. – Williamben Jul 11 '17 at 07:49
  • Your problem is different because the error is about the input shape, not the input being a sparse matrix. – Dr. Snoopy Jul 11 '17 at 08:35

0 Answers0