Read (m x n) comma separated lines of a .txt file

Question

Hello I have these kind of data in a text file and i wanted to read the data inside it.

2003,04,15,15,15,00,38.4279,-76.61,1565,3.7,0.0,38.19,-999,-999,3.9455,3.1457,2.9253

2003,04,15,16,50,00,38.368,-76.5,1566,3.7,0.0,35.01,-999

2003,04,15,17,50,00,38.3074,-76.44

I have used the following codes:

a= zeros(4460,216);
nl = a(:,1);
nc = a(1,:); 

if fid>0
  for i = 1:length(nl)
    d = textscan(Ligne,'%f','whitespace',',');
    numbers = d{:}';    
    D = a(i) + numbers;
    i = i+1;
  end
  Ligne = fgetl(fid);
end

The problem is that i cant implement the matrix D. The data are being replaced each time. Can somebody help me please?

Could you kindly accept an answer or update your question in case you still got problems? — Potaito, Mar 31 '15 at 16:38

score 2 · Accepted Answer · edited May 23 '17 at 12:15

Assuming your file looks like:

Header
Header
Header

2003,04,15,15,15,00,38.4279,-76.61,1565,3.7,0.0,38.19,-999,-999,3.9455,3.1457,2.9253

2003,04,15,16,50,00,38.368,-76.5,1566,3.7,0.0,35.01,-999

2003,04,15,17,50,00,38.3074,-76.44

In the example you have 4 headerlines and the delimiter is ','. Now just use importdata as a very convenient import function:

X = importdata('myData.txt',',',4)

which returns:

X = 

          data: [3x17 double]
      textdata: {4x17 cell}
    colheaders: {1x17 cell}

X.data contains your numeric data. As the data in your file has a different number of entries in every row, missing values are filled with NaN. X.textdata contains the skipped header lines as strings.

You can process them, if needed with textscan:

additionalInformation = textscan(X.textdata, ... )

The alternative suggested by Shai using csvread with the row offset set to 4 does the job as well. But be aware that missing values are replaced with zeros, what I personally dislike for further processing of data. Especially as your actual data also contains zeros.

X = csvread('myData.txt',4)

hello, sorry for late reply. Importdata also works just fine. Thank you for your help. — Sevian, Apr 01 '15 at 06:47

score 0 · Answer 2 · edited May 23 '17 at 11:59

0

Have you considered using csvread?

D = csvread( filename );

Regarding your code, you have two major bugs

D = a(i)+numbers; - you actually override D at each iteration. Try D(i,:) = a(i,:)+numbers; instead
i=i+1; - you change the loop variable inside the loop! if you are using a for-loop on i you do not need to increment it manually.

And some comments:

It is best not to use i as a variable name in Matlab.
You pre-allocated a but not D, consider pre-allocating D as well.

edited May 23 '17 at 11:59

Community

1
1

answered Mar 30 '15 at 08:12

Shai

111,146
38
238
371

Yeah the problem is that i have got few data in string format at the top of the file, thats why im reading line by line. If i use .csv i need to delete the data on top. – Sevian Mar 30 '15 at 08:15
1

@Sevian Have a closer look at the doc. You can skip the header of your file by defining a row offset. – Robert Seifert Mar 30 '15 at 08:17

score 0 · Answer 3 · answered Mar 30 '15 at 08:13

You already said it: D is replaced every time. This is happening since you don't specify indices when accessing D. You should do something like

D = zeros(size(a))
....
if ...
  for ... 
    ...
    D(i) = a(i) + numbers;
    ...
  end
end

But as Shai pointed out, there might be a simpler solution to your problem.

Sevian · Answer 4 · 2015-03-30T09:23:37.160

0

Finally i have used these code lines. D = NaN(size(a)); i=1;

while ~(Ligne==-1)

    d = textscan(Ligne,'%f','whitespace',',');
    numbers = d{:}';    
    D(i,:) = numbers;
    Ligne = fgetl(fid);
    i=i+1;

end

edited Mar 30 '15 at 09:23

answered Mar 30 '15 at 09:17

Sevian

57
7

But for what reason? This code is so slow compared to other answers provided. – Robert Seifert Mar 30 '15 at 10:17
I have approximately 100 unwanted rows of other data before the data i wanted to extract from the .txt file. So i needed an automated code to read all file having this kind of format. I had to read line by line to extract the relevant information needed. I will surely keep your codes it might help in the futur. Thank you so much. – Sevian Mar 30 '15 at 13:23
I understand. But then you should still use `X = importdata( ... )` and then just use textscan for your header with `... = textscan(X.textdata, ...)`. Have a look at my edited answer to see what I mean. It is really not very fast to read all your data line by line, if you could load it at once. – Robert Seifert Mar 30 '15 at 13:54

Read (m x n) comma separated lines of a .txt file

4 Answers4