I am trying to numerically determine the rank of large matrices (around 50000 x 50000 64-bit floating point) using QR factorization with column pivoting. Even when the matrix is somewhat sparse (density ~0.05), I am having trouble completing the factorization before running out of memory (499GB). I am currently doing this in Julia with the SparseArrays package (an interface for the SuiteSparse library), calling qr(A) or rank(A) where A is SparseMatrixCSC{Float64, Int64}. Is there some way to reduce the memory usage since I only need R but not Q? Are there other languages/libraries that could be more suitable for my purpose?
I also looked into other rank-revealing algorithms, but QR with pivoting seems to be the only numerically stable option.
