vector> to mxArray using memcpy

Question

I have correlation matrix of a data and i want to use pca to transform them to uncorrelated set.

so i've decided to use matlab engine(c++ mex API) to perform the pca

my question is how to copy the matrix contents to mxArray efficiently i used loops to allocate each element of matrix...on the other hand i've looked up for memcpy and it seems error prone.

although i've tested the following and it just copies the first column!

memcpy((double *)mxGetPr(T), &rho_mat[0][0], rows * sizeof(double));

what is the best way to copy the data (matrix -> mxArray and mxArray -> matrix) ?

void pca(vector<vector<double>>& rho_mat)
{
    Engine *ep;
    mxArray *T = NULL, *result = NULL;

    if (!(ep = engOpen(""))) {
        fprintf(stderr, "\nCan't start MATLAB engine\n");
        return;
    }

    size_t rows = rho_mat.size();
    size_t cols = rho_mat[0].size();

    T = mxCreateDoubleMatrix(rows, cols, mxREAL);

   double * buf = (double *)mxGetPr(T);

   for (int i = 0; i<rows; i++) {
       for (int j = 0; j<cols; j++) {
           buf[i*(cols)+j] = rho_mat[i][j];
       }
   }

    engPutVariable(ep, "T", T);


    engEvalString(ep, "PC = pcacov(T);");
    result = engGetVariable(ep, "PC");   
}

Thanks
Regards

The trick is to not use a `vector>` in the C++ code. Either get a matrix library or write your own matrix class that use a `vector` and fake that it has multiple dimensions like you do with `buf`. Then you can copy the underlying data much easier. — NathanOliver, Oct 22 '18 at 20:05
matrix lib like eigen? could you please tell me why memcpy just transport first column data? — nauok, Oct 22 '18 at 20:12
Allocating for each row is one of the worst ways to create a matrix. Cut the allocations down to one by using what was mentioned earlier, or to two allocations if you desire the `[][]`syntax. — PaulMcKenzie, Oct 22 '18 at 20:18
@nauok `memcpy` fails because each row of the 2d vector doesn't have to be next to each other in memory. — NathanOliver, Oct 22 '18 at 20:19
@NathanOliver should i use loop to iterate over memory blocks? — nauok, Oct 22 '18 at 20:29
@nauok You need to allocate the entire pool of memory in one call. [See this](https://stackoverflow.com/questions/21943621/how-to-create-a-contiguous-2d-array-in-c/21944048#21944048) — PaulMcKenzie, Oct 22 '18 at 20:30
I edited the link. Please try again. The pool of memory that is the 2d array is allocated once. All the data is contiguous, not disjoint as your code is doing now. — PaulMcKenzie, Oct 22 '18 at 20:35

Sahil Dhoked · Accepted Answer · 2018-10-24T20:48:32.103

2

You can try using std::memcpy in a loop for each row.

for (int i = 0; i<rows; i++)
{
    std::memcpy(buf + i*cols, &rho_mat[i][0], cols * sizeof(double));
}

Please note you have to use cols in you memcpy to ensure each row is copied. In your example, it might have been coincidental if your matrix was square.

You can refer to this answer on how to copy a 1-d vector using memcpy.

Edit:

To copy from 2-D array to 2-D vector(assuming vector is already of size rows*cols)

for (int i = 0; i<rows; i++)
{
    std::memcpy(&rho_mat[i][0], buf + i*cols, cols * sizeof(double));
}

Please note the assumption made

OR

A much cleaner way would be to use std::assign or constructor to std::vector

if(rho_mat.size() == 0)
{
    for (int i = 0; i<rows; i++)
    {
         rho_mat.push_back(vector<int>(buf + i*cols, buf + i*cols + cols));
         //OR
         //rho_mat.push_back(vector<int>());
         //rho_mat[i].assign(buf + i*cols, buf + i*cols + cols);
    }
}

edited Oct 24 '18 at 20:48

answered Oct 22 '18 at 20:34

Sahil Dhoked

372
2
12

1

You can look into std::copy as well – Sahil Dhoked Oct 22 '18 at 20:37
thank you... about the matrix dim , it always be square matrix – nauok Oct 22 '18 at 20:40
is there way not using loops, the matrix may be 150*150, and it will be very time consuming. – nauok Oct 22 '18 at 20:43
Since you are using 2-D vector, each individual 1-D vector will have a different start location and may not necessarily be contiguous, especially if you have large dimensions. Look at this [post](https://stackoverflow.com/questions/42448597/2d-stdvector-contiguous-memory>). If you really want your elements to be contiguous, you can use a 1-D vector and fake the remaining dimensions as mentioned in the comments. – Sahil Dhoked Oct 22 '18 at 20:51
Note that avoiding loops would not give a huge benefit as you expect. Theoretically it would always be O(N^2) complexity. This means that at some level your processor is going to have to make a computation of some fraction of rows*cols. – Sahil Dhoked Oct 22 '18 at 20:55
std::memcpy(&Evec[i][0], out_buf + i * cols, cols * sizeof(double)); i've tried swapping destination and source in memcpy in order to convert mxArray to 2d vector but it doesn't work...could you please help me with this? and it shows me access violation error – nauok Oct 24 '18 at 08:29
Converting array to vector is more complicated than vector to array. You should look into std::copy instead of using std::memcpy – Sahil Dhoked Oct 24 '18 at 20:12
could you please include it to the answer? i really had hard time fixing my problem – nauok Oct 24 '18 at 20:15
Modified the answer. Please let me know if you have any more comments – Sahil Dhoked Oct 24 '18 at 20:49

vector> to mxArray using memcpy

1 Answers1