De-nest elements of cell-matrix into a matrix

Question

EDIT: It turns out this problem is not solved, as it fails to handle empty cells in the source data. i.e. k = {1 2 [] 4 5}; cat( 2, k{:} ) gives 1 2 4 5 not 1 2 NaN 4 5. So the subsequent reshape is now misaligned. Can anyone outline a strategy for handling this?

---

I have data of the form:

X = {  ...
       { '014-03-01' [1.1] [1.2] [1.3] }; ...
       { '014-03-02' [2.1] [2.2] [2.3] }; ...  %etc
    }

I wish to replace [1.1] with 1.1 etc, and also replace the date with an integer using datenum

So I may as well use a standard 2D matrix to hold the result (as every element can be expressed as a Double).

But how to go out this repacking?

I was hoping to switch the dates in-place using X{:,1} = datenum( X{:,1} ) but this command fails.

I can do:

A = cat( 1, X{:} )

dateNums = datenum(  cat( 1, A{:,1} )  )
values = reshape(  cat( 1, A{:,2:end} ),  size(X,1),  []  )

final = [dateNums values]

Well, that works, but I don't feel at all comfortable with it.

>> u = A{:,1}
u =
    014-03-01
>> cat(1,u)
ans =
    014-03-01

This suggests only one value is output. But:

>> cat(1,A{:,1})
ans =
    014-03-01
    014-03-02

So A{:,1} must be emitting a sequential stream of values, and cat must be accepting varargs.

So now if I do A{:,2:end}, it is now spitting out that 2D subgrid again as a sequential stream of values...? And the only way to get at that grid is to cat -> reshape it. Is this a correct understanding?

I'm finding MATLAB's console output infuriatingly inconsistent.

Answer 1

The "sequential stream of values" is known as a comma-separated list. Doing A{:,1} in MATLAB in the console is equivalent to the following syntax:

>> A{1,1}, A{2,1}, A{3,1}, ..., A{end,1}

This is why you see a stream of values because it is literally typing out each row of the cell for the first column, separated by commas and showing that in the command prompt. This is probably the source of your infuriation as you're getting a verbose dump of all of the contents in the cell when you are unpacking them into a comma-separated list. In any case, this is why you use cat because doing cat(1, A{:,1}) is equivalent to doing:

cat(1, A{1,1}, A{2,1}, A{3,1}, ... A{end,1})

The end result is that it takes all elements in the 2D cell array of the first column and creates a new result concatenating all of these together. Similarly, doing A{:, 2:end} is equivalent to (note the column-major order):

>> A{1, 2}, A{2, 2}, A{3, 2}, ..., A{end, 2}, A{1, 3}, A{2, 3}, A{3, 3}..., A{end, 3}, ..., A{end, end}

This is why you need to perform a reshape because if you did cat with this by itself, it will only give you a single vector as a result. You probably want a 2D matrix, so the reshape is necessary to convert the vector into matrix form.

Comma-separated lists are very similar to Python's splat operator if you're familiar with Python. The splat operator is used for unpacking input arguments that are placed in a single list or iterator type... so for example, if you had the following in Python:

l = [1, 2, 3, 4]
func(*l)

This is equivalent to doing:

func(1, 2, 3, 4)

The above isn't really necessary to understand comma-separated lists, but I just wanted to show you that they're used in many programming languages, including Python.

Answer 2

There is a problem with empty cells: cat will skip them. Which means that a subsequent reshape will throw a 'dimension mismatch' error.

The following code simply removes rows containing empty cells (which is what I require) as a preprocessing step.

(It would only take a minor alteration to replace empty cells with NaNs).

A = cat( 1, X{:} );

% Remove any row containing empty cells
emptiesInRow = sum( cellfun('isempty',A), 2 );
A( emptiesInRow > 0, : ) = [];

% Date is first col
dateNums = datenum(  cat( 1, A{:,1} )  );

% Get other cols
values = reshape(  cat( 1, A{:,2:end} ),  size(A,1),  []  );

% Recombine into (double) array
grid = [dateNums values];  %#ok<NASGU>