0

The scenario is I need to load data, which is in the form of struct array, from several .mat files and combine them into one single array for later processing. Initially what I did was like:

raw_data = [];
for i=1:length(file_name_list)
    raw_data = [raw_data importdata(file_name_list(i))];
end

Then MATLAB warns me that:

The variable 'raw_data' appears to change size on every loop iteration. Consider preallocating for speed.

If I understand correctly, when appending to the array, MATLAB would reallocate some space and copy all the original elements to the new place so it might take a lot of time. However the question is that I don't know in advance how many struct will be in the files and I don't know the size of each struct. Is there a way to speed up the process? It already takes 5 seconds to finish 2 files with 600 struct in total (which is very slow).

Thank you for your help!

AppleJ
  • 1
  • You might need to take a look at http://www.mathworks.com/help/matlab/matlab_prog/techniques-for-improving-performance.html – CroCo Jul 17 '14 at 08:09

1 Answers1

0

Try the following:

Initialize raw_data as a cell array of size equivalent to the number of .mat files.

raw_data=cell(length(file_name_list),1);
for i=1:length(file_name_list)
    raw_data{i} = importdata(file_name_list(i));
end

This should solve your problem.

Naveen
  • 306
  • 1
  • 6
  • 1
    Just a small remark: it is best [not to use `i` as a variable name in Matlab](http://stackoverflow.com/questions/14790740/using-i-and-j-as-variables-in-matlab). – Shai Jul 17 '14 at 08:38
  • @ Shai Thank you. Shall make a note of it. – Naveen Jul 17 '14 at 08:42
  • Thank you Naveen and Shai. If I know the struct field in advance, do you think if I initial raw_data as a large struct array before for loop will help? For example 'raw_data=repmat(struct('field1',[],'field2',[]),1,1000000)'. Actually I was not really clear about how MATLAB store struct, if the fields can be any data type arbitrary large, how do they allocate memory for struct variable? – AppleJ Jul 18 '14 at 23:05