0
sc_f = sc27.tocsr()
X, y = sc_f
X_train,X_test, y_train, y_test = train_test_split(sc_f, y, stratify=y, test_size=0.3)

I was getting error while splitting the sparse matrix to a Train and Test. How to split the array into train and test?

sc_f: scipy sparse matrix

I want to split it in such a way so that in y I have the target i.e 'is_true' which I want to predict and it takes only two values 0 and 1. When I'm converting my scipy sparse matrix to sparseDataFrame. I'm not able to access it as df['is_true'] where df is the sparseDataFrame. How do I split it. Can anyone help?

Dhruv Bhardwaj
  • 11
  • 6
  • You need to describe the error. We can't guess. Also make sure that `train_test_split` can work with sparse matrix, and if so, are there any restrictions. Any questions about `sparseDateFrame` belong in another question. – hpaulj Jun 23 '18 at 16:20
  • I wan to split my Sparse Matrix such that X = np.array(sc_f), y = np.array(sc_f['is_true']). Is there some way to split in such a way a sparse matrix? I want my target 'y' to contain 'is_true'. Can you tell how to do this? – Dhruv Bhardwaj Jun 23 '18 at 16:26
  • What is `X, y = sc_f` supposed to do? With my examples that produces an error. The `x['is_true']` expression only makes sense is `x` is a dictionary (or a few other objects that take a string index). That certainly doesn't apply to `csr` sparse matrices. `arr = sc_f.toarray()` will create a dense `ndarray` that might be easier to work with, at least at the learning stage. Also read [mcve] – hpaulj Jun 23 '18 at 17:55
  • While converting to dense matrix getting a memory error. – Dhruv Bhardwaj Jun 23 '18 at 18:11
  • 1
    If you get a memory error, that isn't a minimal example. – hpaulj Jun 23 '18 at 20:00

0 Answers0