4

This is more of a Matlab programming question than it is a math question.

I'd like to run gradient descent multiple on different learning rates. I have a set of learning rates 

alpha = [0.3, 0.1, 0.03, 0.01, 0.003, 0.001];

and each time I run gradient descent, I get a vector J_vals as output. However, I don't know Matlab well enough to know how to implement this besides doing something like:

[theta, J_vals] = gradientDescent(...., alpha(1),...);
J1 = J_vals;
[theta, J_vals] = gradientDescent(...., alpha(2),...);
J2 = J_vals;

and so on.

I thought about using a for loop, but then I don't know how I would deal with the J_vals's (not sure how to apply the for loop to J1, J2, and so on). Perhaps it would look something like this:

for i = len(alpha)
     [theta, J_vals] = gradientDescent(..., alpha(i),...);
     J(i) = J_vals;
end

Then I would have a vector of vectors.

In Python, I would just run a for loop and append each new result to the end of a list. How do I implement something like this in Matlab? Or is there a more efficient way?

TheRealFakeNews
  • 7,512
  • 16
  • 73
  • 114

3 Answers3

3

If you know how many loops you are going have and the size of the J_vals (or at least a reasonable upper bound) I would suggest pre-allocating the size of the container array

J = zeros(n,1);

then on each loop insert the new values

J(start:start+n) = J_vals

That way you don't reallocate memory. If you don't know, you can append the values to the array. For example,

J = []; % initialize
for i = len(alpha)
     [theta, J_vals] = gradientDescent(..., alpha(i),...);
    J = [J; J_vals]; % Append column row
end

but this is re-allocating the size of the array every loop. If it's not too many loops then it should be ok.

dpmcmlxxvi
  • 1,292
  • 1
  • 9
  • 13
3

Matlab's "cell arrays" are kind of like lists in Python. They are similar in that you can put variable datatypes into them. Nobody seems to be too sure, but most likely the cell array is implemented as an array of object pointers. That means that it is still somewhat expensive to append to it (cell_array{length(cell_array) + 1} = new_data), but at least you are only appending a pointer instead of the entire column. You would still have to convert the cell array to a normal matrix afterward using cell2mat.

The most idiomatic Matlab solution is to pre-allocate (as @dpmcmlxxvi suggested).

I think what you are describing is a really common use case, and it's unfortunate that Matlab requires such a verbose idiom for this. Also it's frustrating that the documentation is opaque on how cell arrays are implemented and whether it is expensive to append to a cell array.

Community
  • 1
  • 1
cxrodgers
  • 4,317
  • 2
  • 23
  • 29
2

Your solution works just fine as long as you add a : for the row subscript (assuming J_vals is a column vector):

for i = len(alpha)
     [theta, J_vals] = gradientDescent(..., alpha(i),...);
     J(:, i) = J_vals;
  %//  ^... all rows, column 'i'
end

You could even put that as the return value:

for i = len(alpha)
     [theta, J(:, i)] = gradientDescent(..., alpha(i),...);
     %//       ^... add returned value directly to our list
end

Both of these methods allow you to preallocate your matrix for a potential speed gain.

If you want to build your list as you go, you can use the method in @dpmcmlxxvi's answer, or you can use the special subscript end. Neither of these methods are compatible with preallocation, though.

for i = len(alpha)
     [theta, J(:, end+1)] = gradientDescent(..., alpha(i),...);
     %//          ^... add new vector after the current end of list
end

I would also like to suggest you not use i as a variable name in Matlab. I know it's natural for other languages, but in Matlab it overwrites the built-in imaginary constant i.
See: https://stackoverflow.com/a/14790765/1377097

Community
  • 1
  • 1
beaker
  • 16,331
  • 3
  • 32
  • 49