Loop through files in a folder in matlab

Question

I have a set of days of log files that I need to parse and look at in matlab.

The log files look like this:

LOG_20120509_120002_002.csv
(year)(month)(day)_(hour)(minute)(second)_(log part number)

The logs increment hourly, but sometimes the seconds are one or two seconds off (per hour) which means i need to ignore what they say to do loadcsv.

I also have another file:

LOG_DATA_20120509_120002.csv

which contains data for the whole hour (different data).

The overall objective is to:

 loop through each day 
     loop through each hour
         read in LOG_DATA for whole hour
         loop through each segment
             read in LOG for each segment
                 compile a table of all the data

I guess the question is then, how do i ignore the minutes of the day if they are different? I suspect it will be by looping through all the files in the folder, in which case how do i do that?

Isaac · Accepted Answer · 2013-09-03T19:33:40.187

104

Looping through all the files in the folder is relatively easy:

files = dir('*.csv');
for file = files'
    csv = load(file.name);
    % Do some stuff
end

edited Sep 03 '13 at 19:33

answered Jul 23 '12 at 23:15

Isaac

3,586
1
18
20

I had a feeling it would be something with dir, i couldn't quite put my finger on it though. Thanks for your response. – Fantastic Mr Fox Jul 23 '12 at 23:18
2

mmm this is not working for me... file = files simply copies files into file :( – Juan Sebastian Totero Feb 25 '13 at 09:29
12

Try `file = files'`; it may require that `files` is a row-array. – Isaac Feb 26 '13 at 22:43
1

@Isaac Keep in mind that `load` is designed for use with files containing MATLAB variables only, so even though it will work for CSV files that are completely numeric, if the header row contains variable names or the data are more complicated, it may be better to use [`textscan`](http://www.mathworks.com/help/matlab/ref/textscan.html) or [`csvread`](http://www.mathworks.com/help/matlab/ref/csvread.html). – John Bensin Jul 22 '13 at 13:46
@JohnBensin I know this is late but that is not entirely true. `load` can get space separated data out of files and other specific things. – Fantastic Mr Fox Jun 25 '14 at 22:30
Works good. However, one potential **pitfall**: pay attention to the file order as a result of the file names, if the file order matters. I spent hours debugging and realized this. _[facepalm]_ – Sibbs Gambling Jul 09 '14 at 14:44
2

When I do this, I get an error saying `Unable to read file 'abcdef.csv' : ': no such file or directory.` why is that? – vigamage Oct 15 '15 at 04:07
1

@vigamage, are you targeting the current directory? Or are you using `dir('some/path/*.csv')`? – Isaac Oct 15 '15 at 16:29
why do you need the `'` (transpose)? – Charlie Parker Feb 03 '16 at 05:45
@CharlieParker, because matlab for-loops treat row-vectors and column-vectors differently (or at least they did when I wrote this). – Isaac Feb 05 '16 at 17:35
how do you actually deal with this without having to cd to a specific directory? I have a directory with more directories with those directory having files. It seems really dum to have to cd to directories then call `dir`, is there not a better solution? – Charlie Parker Mar 30 '18 at 17:57
@CharlieParker, you can call `dir` directly on a subdirectory to get the contents of that subdirectory. You can also loop through the names returned by `dir`, and call `isdir` on each one to figure out if they are directories themselves. In this way you can recursively list all the files in a nested directory. – Isaac Apr 02 '18 at 13:54

score 3 · Answer 2 · answered Jun 25 '18 at 05:22

3

At first, you must specify your path, the path that your *.csv files are in there

path = 'f:\project\dataset'

You can change it based on your system.

then,

use dir function :

files = dir (strcat(path,'\*.csv'))

L = length (files);

for i=1:L
   image{i}=csvread(strcat(path,'\',file(i).name));   
   % process the image in here
end

pwd also can be used.

answered Jun 25 '18 at 05:22

PyMatFlow

459
4
8

2

Instead of `strcat`, use `fullfile`: `dir(fullfile(path,'*.csv'))`. See documentation: https://www.mathworks.com/help/matlab/ref/fullfile.html – Cris Luengo Jun 25 '18 at 06:07

Loop through files in a folder in matlab

2 Answers2

Linked

Related