How to Read csv file with line breaks when "\r\n" does not work

Question

I am new to Java and have been reading Java docs and other threads (1 ,2) but couldn't make it work.

Basically my csv file has few records which read like this

How are
you

so I want my code to read it as one line

How are you

My code looks like this:

BufferedReader bReader = new BufferedReader(new InputStreamReader(new FileInputStream(csv),"utf-8"));

    while ((line = bReader.readLine()) != null) {
        String lines = line.replaceAll("\r\n", " ");
        System.out.println(lines);

Manually, when I pressed backspace at youit goes back with areand I pressed space. Then it was fine. But I have a big csv file with 29k records. There must be a way through which I can fix this. Can you please point me towards the direction? Thank you.

[Edit]

This is how it appears.

Fav: Beaver tails.
Least fav: HST not included in prices.

Edit 2:

-3166,1054,CF ,5992841,15:37.5,en,13007,12,12,Comments: Favorite and/or least favorite things,0,"Cafe Fun
-Least favourite - cabs"

"Cafe Fun Least Favourite - cabs" should be on the same line.

Processing CSV correctly is tricky, especially when dealing with quotes fields. Don't write your own CSV reader, use one of the existing, _debugged_ CSV libraries. — Jim Garrison, Jan 22 '18 at 19:14
Can you paste a peice that would actually be found in your file, eg. "is this\r\na record", but is "this"\r\n"two records" ? — matt, Jan 22 '18 at 19:22
I don't see any comma's. I also don't see how this relates to newlines being part of the same record. — matt, Jan 22 '18 at 19:32
@matt where I open the csv file in excel at this specific cell, I have to extend the cell read it because it is one step down.If I am using sublime text reader, then it is showing a new line — Tam, Jan 22 '18 at 19:34
Jim Garrison has the correct idea. I've had good results using Apache Commons CSV. Never tried OpenCSV. Please ignore (or downvote) any answers here that don't use a CSV library - you shouldn't be re-inventing wheels. — Dawood ibn Kareem, Jan 22 '18 at 19:40
I mean you need to show the actual csv you want to parse. Not the table that you see in excel. If you have some subset of csv that you want to parse, then *maybe* you can get away with what you're doing. From the sounds of it though, you should get a csv reader. Do you have cells like "Hey, I am a csv cell with a comma." — matt, Jan 22 '18 at 20:36
@matt I have cells like that. I am going to take a crack at Apache Commons csv and give it a try. — Tam, Jan 23 '18 at 13:21

score 1 · Answer 1 · edited Jan 22 '18 at 19:14

1

readLine() will return the next line in the file, without the line separator. So on the first iteration of your loop, lines is "How are" and on the second iteration, lines is "you". Neither of these contain "\r\n", so your calls to replaceAll(...) just return the same string.

Then, System.out.println(...) prints the text with a line separator appended, so you get back to what you started with.

You can collect all the lines into a list:

List<String> lines = Files.readAllLines(csv);

and then concatenate them using String.join(...):

String allLines = String.join(" ", lines);

edited Jan 22 '18 at 19:14

Edwardth

697
1
4
14

answered Jan 22 '18 at 19:10

James_D

201,275
16
291
322

@Sweeper In that case, yes. I read "my file has few records" as meaning that the file had only a few lines. – James_D Jan 22 '18 at 19:17
@Sweeper "readLine" could load the whole file into memory and cause an out of memory error too. – matt Jan 22 '18 at 19:25

score 0 · Answer 2 · answered Jan 22 '18 at 19:07

BufferedReader.readLine() doesn't read the newline, so your String lines does't have a brake.
You only print a newline with System.out.println(lines); change it to System.out.print(lines); and invoke System.out.println(); after the while-loop.

BufferedReader bReader = new BufferedReader(new InputStreamReader(new FileInputStream(csv),"utf-8"));

while ((line = bReader.readLine()) != null) {
    System.out.print(line);
}
System.out.println();

score 0 · Answer 3 · answered Jan 22 '18 at 19:18

To start the csv (as the name implies) are files separated by commas, not by spaces. But forgetting that, the readLine only reads the line where the "pointer" is, and in that case "you" is in another line than "How are". I think that's where your problem lies. One way to solve it would be to use the StringBuilder and the "append (String)". and it is adding everything together. Regards

How to Read csv file with line breaks when "\r\n" does not work

3 Answers3