Questions tagged [bigstatsr]
11 questions
2
votes
0 answers
How to create a high dimensional array using FBM in the package bigstatsr?
The FBM function in bigstatsr package allows us to create a 2D matrix with shared memory. For instance, FBM(5, 8) will create a 5 by 8 matrix using shared memory.
However, is there a way to create a 3D matrix with shared memory in a similar…
Caprikuarius
- 176
- 7
2
votes
1 answer
Convert bigstatsr::FBM to matrix?
Is is possible to convert an bigstatsr::FBM object to a regular R matrix?
For example, suppose that I generate the following matrix:
N <- 100
K <- 50
`%dopar%` <- foreach::`%dopar%`
`%:%` <- foreach::`%:%`
mat3 <- bigstatsr::FBM(N, K)
cl <-…
Ignacio
- 7,646
- 16
- 60
- 113
1
vote
1 answer
How to populate bigstatsr::FBM with sqlite database for later consumption?
I'm a newbie to the bigstatsr package. I have a sqlite database which I want to convert to an FBM matrix of 40k rows (genes) 60K columns (samples) for later consumption. I found examples of how to populate the matrix with random values but I'm…
Rec
- 37
- 4
1
vote
1 answer
FBM() of bigstatsr does not calculate the matrix correctly while using parallel foreach as it does when the code is running in a simple for loop
I need to estimate a transition Matrix. Since I have lots of data I tried to run it parallel using foreach and I tried the shared-memory function FBM() of bigstatsr. And it seems that the function does not all the time return the right result.…
9001_db
- 81
- 10
1
vote
1 answer
store results from a nested foreach loop in a FBM class
I am working with a big shared memory matrix of 1.3e6x1.3e6 in a foreach loop. I create that matrix with FBM function of bigstatsr package.
I need the results of the loop in the FBM class object to not run out of RAM memory.
This is what I want to…
LauC
- 55
- 5
1
vote
0 answers
FBM function for a sparse matrix of 1.3e6x200
I need a memory shared sparse matrix of 1.3e6x200. I have tried with bigmemory package but my computer ran out of memory. I have found the function FBM in the package bigstatsr. My code run pretty good with this function for a small example, but I…
LauC
- 55
- 5
1
vote
1 answer
How can I subset rows or columns of a bigstatsr::FBM in Rcpp and store them in a vector?
I have a function that computes basic summary statistics from the rows (or columns) of a given Matrix and I am now trying to also use this function with a bigstatsr::FBM (I am aware that using columns should be more efficient).
The reason I want to…
Nikos Bosse
- 144
- 11
0
votes
0 answers
subsetting from an bigstatsr::FBM object for LDpred2 R-tutorial
I'm using LDpred2 incorporated in bigsnpr to calculate polygenic scores with my own set of genetic Data. I am following the steps found in the online tutorial of LDpred2 on Github (https://privefl.github.io/bigsnpr/articles/LDpred2.html) to use the…
0
votes
1 answer
How to use the bigstatsr R package using two datasets for estimating the parameters?
I have independent and dependent datasets. I want to test all possible relationships between dependent and independent variables. In my previous post (How to replicate a function using mapply with multiple arguments to calculate the power of a…
user8129108
- 5
- 5
0
votes
0 answers
Capturing parallelized R output types: Alternatives to bigstatsr that supports tibble like matrice and different variable types?
Previously I used bigstatsr to store computed values (double).
These values were computed in parallel using the doParallel package.
Now, I'm trying to use bigstatsr to store more complex objects. Typically in the form of list.
And only now I notice…
Afiq Johari
- 1,372
- 1
- 15
- 28
0
votes
1 answer
Problem with big_cprodMat output: Matrix too large
I am trying to get the cross-product of two "Large matrices" in R. Unfortunately, the data cannot be disclosed here, but the dimensions are 177304*23, and 23*177304, so that the results should have the dimensions 177304*177304.
As my RAM is 8 GB, I…
