What is the best way to convert Matlab multidimensional cell array to Javascript array?

Question

I have a very large 4D Matlab matrix (31x31x86x127) that I wish to convert into a Javascript 4D array. What is the best way to do this?

Currently my tentative approach will be to either:

1) Write the Matlab matrix into a binary file, and then read that in and build the Javascript.

2) Use JSONlab (http://www.mathworks.com/matlabcentral/fileexchange/33381-jsonlab--a-toolbox-to-encode-decode-json-files-in-matlab-octave) to convert the Matlab matrix into a JSON string and then write a custom decoder to turn that JSON string into a Javascript Array. Issue is that the JSON text file is 1.98GB...

3) This may be the best way.

fileID = fopen('test.bin', 'w'); fwrite(fileID,value,'double');

Test.bin is then around 82MB, which is actually what I expect. 31*31*86*127*8bits/double = 82ish MB! However, how do I then read (in the browser) this binary file to a 4d Javascript array? Thanks! Thoughts?

Thanks for your help!

There is something wrong with the numbers. According to your numbers, it's 630bytes/number. I know that json is inefficient, but it seems there are hundreds of blanks in it — Daniel, Sep 01 '15 at 07:20
The json string looks like this: ```"testoutput": { "_ArrayType_": "double", "_ArraySize_": [31,31,86,127], "_ArrayData_": [-0.08103847277,-0.08300451255,-0.08514257385,-0.08685153974,.... etc``` — txizzle, Sep 01 '15 at 08:25

Daniel · Accepted Answer · 2015-09-01T16:26:57.520

1

save is not the right function to write a text file. Use savejson or saveubjson and pass the filename to the function. Do not use the return argument of these functions. Doing so I get a ubjson with less than 100MB and a json with less than 150MB.

My original answer, based on insufficient knowledge about the used code:

Instead of writing your own binary format, use one of the already available binary formats. Try writing it to universal binary json, jsonlab does support it. you should end up with a reasonable sized data without losing the advantages of a standardized file exchange format.

edited Sep 01 '15 at 16:26

answered Sep 01 '15 at 07:34

Daniel

36,610
3
36
69

Using ubjson, the JSON file is still 1.3gb... by my calculations, I should need only 31*31*86*127*8bits/double = 83mb. Am I missing something?? These are some sample entries: 0.08103847277,-0.08300451255,-0.08514257385,-0.08685153974 – txizzle Sep 01 '15 at 08:22
There is something wrong. Can you provide example code to reproduce your issue? Start with `rand` or similar – Daniel Sep 01 '15 at 09:44
>>size(value) returns ans = 31 31 86 127. >>ubjson = saveubjson('foo', value); >>save('value_ubjson.txt', 'ubjson', '-ascii');. Maybe it's because I am saving in ASCII that makes the file size so large? – txizzle Sep 01 '15 at 15:18
Hm. By "do not use the return argument of these functions", I assume you mean that I shouldn't save the output of saveubjson. However, when I do ```>>saveubjson('filename.txt', value);```, nothing is saved. Is this what you mean? Nothing is saved for both the cases where I create an empty 'filename.txt' and when I don't create one. – txizzle Sep 01 '15 at 16:43
Read the documentation, first input argument is the name of the root, not the name of the file. – Daniel Sep 01 '15 at 16:46
1

Ignore above comment! Thanks. This worked: ```>>saveubjson('root name', value, 'filename.txt');```. Thanks for the help! – txizzle Sep 01 '15 at 16:54

score 0 · Answer 2 · edited May 23 '17 at 12:06

I think the best way is to

Write the matrix out as a string or text file (binary file is not necessary). You will need n-1 delimiters, where n=4 is the number of dimensions for your case. See this Saturn Fiddle as an example for a 2D matrix. Code below
Read the text file into a JavaScript string. How you do this really depends on if you're using JavaScript on the server or web browser.
Parse the string into a JavaScript array. You will have to use the split function on the delimiters from (1) and then enter them into an array like this example.

Code part (1):

% Welcome to SaturnAPI!
% Start collaborating with MATLAB-Octave fiddles and accomplish more.
% Start your script below these comments.
A = [ 1 2 3 ; 4 5 6 ; 7 8 9 ]
for ii=1:size(A)(1)
    for jj=1:size(A)(2)
        printf(" %d ", A(ii,jj));
  end
  printf(";");
end

Code part (3):

function make(dim, lvl, arr) {
  if (lvl === 1) return [];
  if (!lvl) lvl = dim;
  if (!arr) arr = [];
  for (var i = 0, l = dim; i < l; i += 1) {
    arr[i] = make(dim, lvl - 1, arr[i]);
  }
  return arr;
}

var myMultiArray = make(4);

Thanks for the input! I think an issue is file size actually (See edit - when converting to JSON, so ASCII, the file size was a whopping 1.98GB, which is similar to a text file...) — txizzle, Sep 01 '15 at 07:16

What is the best way to convert Matlab multidimensional cell array to Javascript array?

2 Answers2