Questions tagged [sparse-matrix]

A sparse matrix is a data structure in which not every entry is explicitly represented. Related are sparse matrix algorithms and data structures, along with questions about implementation and analyses.

Sparse matrices are matrices with very few non-zero entries. For some cases, sparse matrices refer to matrices with few entries and the rest of the matrix remains undefined (e.g. NaN or NA). This has mathematical and computational implications.

See this Wikipedia entry for a brief overview of concepts.

The mathematical impact may be that the matrix is not invertible, which can impact a variety of algorithms. Data structures are usually affected, as representation of sparse matrices may be more compact than for a dense matrix. Finally, algorithms may be affected, as they may be optimized for application to just the non-zero entries, rather than perform calculations over all entries.

Each of these areas -- math, data structures, and algorithms -- affects the developers, researchers, and users involved in working with sparse matrices.

One scientific software supporting sparse matrices is . It has a compact representation of sparse matrices, and the ability to utilize this property for efficient computations. For more information see Matlab documentation on sparse matrices.

The scientific software for statistic computing and graphics supports sparse matrices via Matrix package. This package comes with any R distribution and can be loaded by library(Matrix) in an R session. See its package manual for more.

4236 questions
199
votes
7 answers

SparseArray vs HashMap

I can think of several reasons why HashMaps with integer keys are much better than SparseArrays: The Android documentation for a SparseArray says "It is generally slower than a traditional HashMap". If you write code using HashMaps rather than…
Paul Boddington
  • 37,127
  • 10
  • 65
  • 116
112
votes
4 answers

How to transform numpy.matrix or array to scipy sparse matrix

For SciPy sparse matrix, one can use todense() or toarray() to transform to NumPy matrix or array. What are the functions to do the inverse? I searched, but got no idea what keywords should be the right hit.
Flake
  • 4,377
  • 6
  • 30
  • 29
106
votes
7 answers

Are Javascript arrays sparse?

That is, if I use the current time as an index into the array: array[Date.getTime()] = value; will the interpreter instantiate all the elements from 0 to now? Do different browsers do it differently? I remember there used to be a bug in the AIX…
Berry
  • 1,719
  • 2
  • 11
  • 15
88
votes
6 answers

sparse 3d matrix/array in Python?

In scipy, we can construct a sparse matrix using scipy.sparse.lil_matrix() etc. But the matrix is in 2d. I am wondering if there is an existing data structure for sparse 3d matrix / array (tensor) in Python? p.s. I have lots of sparse data in 3d and…
zhongqi
  • 1,127
  • 2
  • 11
  • 11
72
votes
7 answers

Sparse matrices / arrays in Java

I'm working on a project, written in Java, which requires that I build a very large 2-D sparse array. Very sparse, if that makes a difference. Anyway: the most crucial aspect for this application is efficency in terms of time (assume loads of…
DanM
  • 7,037
  • 11
  • 51
  • 86
60
votes
5 answers

How can I optimise this use of Python dictionaries?

Question: I've profiled my Python program to death, and there is one function that is slowing everything down. It uses Python dictionaries heavily, so I may not have used them in the best way. If I can't get it running faster, I will have to…
Adam Nellis
  • 1,500
  • 1
  • 16
  • 23
55
votes
7 answers

scipy csr_matrix: understand indptr

Every once in a while, I get to manipulate a csr_matrix but I always forget how the parameters indices and indptr work together to build a sparse matrix. I am looking for a clear and intuitive explanation on how the indptr interacts with both the…
Tanguy
  • 3,124
  • 4
  • 21
  • 29
55
votes
2 answers

Generating a dense matrix from a sparse matrix in numpy python

I have a Sqlite database that contains following type of schema: termcount(doc_num, term , count) This table contains terms with their respective counts in the document. like (doc1 , term1 ,12) (doc1, term 22, 2) . . (docn,term1 , 10) This matrix…
user2374515
53
votes
3 answers

Scipy sparse... arrays?

So, I'm doing some Kmeans classification using numpy arrays that are quite sparse-- lots and lots of zeroes. I figured that I'd use scipy's 'sparse' package to reduce the storage overhead, but I'm a little confused about how to create arrays, not…
spitzanator
  • 1,877
  • 4
  • 19
  • 29
52
votes
2 answers

LCP with sparse matrix

I indicate matrices by capital letters, and vectors by small letters. I need to solve the following system of linear inequalities for vector v: min(rv - (u + Av), v - s) = 0 where 0 is a vector of zeros. where r is a scalar, u and s are vectors,…
FooBar
  • 15,724
  • 19
  • 82
  • 171
52
votes
6 answers

Iterating through a scipy.sparse vector (or matrix)

I'm wondering what the best way is to iterate nonzero entries of sparse matrices with scipy.sparse. For example, if I do the following: from scipy.sparse import lil_matrix x = lil_matrix( (20,1) ) x[13,0] = 1 x[15,0] = 2 c = 0 for i in x: print…
RandomGuy
  • 1,658
  • 4
  • 18
  • 21
49
votes
1 answer

Concatenate sparse matrices in Python using SciPy/Numpy

What would be the most efficient way to concatenate sparse matrices in Python using SciPy/Numpy? Here I used the following: >>> np.hstack((X, X2)) array([ <49998x70000 sparse matrix of type '' with 1135520 stored…
PascalVKooten
  • 20,643
  • 17
  • 103
  • 160
47
votes
3 answers

How to elementwise-multiply a scipy.sparse matrix by a broadcasted dense 1d array?

Suppose I have a 2d sparse array. In my real usecase both the number of rows and columns are much bigger (say 20000 and 50000) hence it cannot fit in memory when a dense representation is used: >>> import numpy as np >>> import scipy.sparse as…
ogrisel
  • 39,309
  • 12
  • 116
  • 125
46
votes
2 answers

Using sparse matrices with Keras and Tensorflow

My data can be viewed as a matrix of 10B entries (100M x 100), which is very sparse (< 1/100 * 1/100 of entries are non-zero). I would like to feed the data into into a Keras Neural Network model which I have made, using a Tensorflow backend. My…
SRobertJames
  • 8,210
  • 14
  • 60
  • 107
43
votes
3 answers

How do I transform a "SciPy sparse matrix" to a "NumPy matrix"?

I am using a python function called "incidence_matrix(G)", which returns the incident matrix of graph. It is from Networkx package. The problem that I am facing is the return type of this function is "Scipy Sparse Matrix". I need to have the…
Mr.Boy
  • 615
  • 1
  • 7
  • 13
1
2 3
99 100