c++ fstream getline function adds \r to the line size

Question

When I run my code on Clion I dont get this error. But when i run my code on my school's dev server, I realize every line size except the last line in my file below has a size line.size()+1.

This is my file

6
5 8 2 5 6 7
5 4 7 3 2 1
2 5 3 4 7 5
6 5 8 7 2 3
6 4 3 1 1 2
2 7 8 2 1 6
2 1
1 2
2 4
4 0
5 3
2 2

It reads first line size as 2 etc. It reads every line size as size +1 except last line.

And this is my code.

#include <iostream>
#include <fstream>
#include <string>
#include <sstream>
#include <vector>
using namespace std;
int num;
vector <vector <int> >board;
int points = 0;
int main(int argc, char **argv) {
    string file_name_2 ;
    vv << argv[2];
    vv >> file_name_2;
    stringstream vd;
fstream myFile;
    myFile.open(file_name_2, ios::in);
    if (myFile.is_open()) {
        string line;
        int index = 0;
        while(getline(myFile, line)){
            if (line.size()== 1 || line.size()==2 || line == "100"){
                stringstream zz;
                zz << line;
                zz >> num;
                .
                .
                .

I am using getline(file_name, line) to read lines. What causes this and how can i fix it?

Please provide a [mre], which includes a function `main` and all `#include` directives. — Andreas Wenzel, Nov 07 '21 at 00:44
@Yksisarvinen I am using Windows with no problem then run this on my university's dev server and have faced with this problem but i actually don't know what is it a dev server. — aysu kaplan, Nov 07 '21 at 00:51
How are you uploading the text file to the university's server? Do you have the option of uploading in text mode instead of binary mode? If you do, then you should use that option. If you don't have that option, can you maybe run the [`dos2unix`](https://linux.die.net/man/1/dos2unix) command on the university's server? — Andreas Wenzel, Nov 07 '21 at 00:57
@AndreasWenzel I just copy paste or drop the file. I think its uploading as txt as it is. — aysu kaplan, Nov 07 '21 at 01:01
The next step to debugging this would be to print the actual contents of a line whose length isn't what you expect. Print each character such that invisible characters aren't left out and go from there. — chris, Nov 07 '21 at 01:06
@aysukaplan: If you do not have access to the `dos2unix` utility on the server, you may also want to try using `sed` to remove the carriage return characters on the server (preserving only the newline characters). See [this link](https://qualitestgroup.com/insights/technical-hub/how-to-remove-windows-carriage-returns-for-text-files-in-linux/) for further information. The problem probably is that on Windows, lines in text files are delimitered by "\r\n" (carriage return followed by line feed), whereas on Linux, they are simply delimitered by "\n" (only line feed). — Andreas Wenzel, Nov 07 '21 at 01:08
@aysukaplan any issues because of [this](https://stackoverflow.com/questions/21567291/why-does-stdgetline-skip-input-after-a-formatted-extraction) maybe? Post a [mcve] as required here. Everybody hates guesswork like that here. [Edit] your question quickly, otherwise it gets closed an deleted. — πάντα ῥεῖ, Nov 07 '21 at 01:09
@chris When i print the lines they printed as it should be. With no extra spaces or antyhing to make the size size+1 — aysu kaplan, Nov 07 '21 at 01:17
@aysukaplan Invisible characters won't necessarily show up as spaces. When looking for invisible characters (such as `\r`), you often need to cast characters to `int`. You could do this for every character, but it might be enough to do `std::cout << static_cast(line[line.size()-1]) << '\n';` (once you've established that `line` is not empty. Your output from this is probably going to be `13`. — JaMiT, Nov 07 '21 at 01:22
If you don't have access to `dos2unix` or `sed` on the server, then you can also remove the carriage return characters from the file on Windows, using your own program, for example with this simple program: `#include int main(void) { int c; while ( ( c = getchar() ) != EOF ) if ( c != '\r' ) putchar( c ); }` You can run this program like this: `programname unixtext` from cmd.exe under Windows. This will create a file called `unixtext` using Linux/UNIX line delimiters. You can then upload that file to the server. — Andreas Wenzel, Nov 07 '21 at 01:22
@AndreasWenzel using sed command solved my problem. For preventing experience the same problem in future, does this problem caused because of how i copied my file? — aysu kaplan, Nov 07 '21 at 01:25
@aysukaplan Copying via Windows Explorer (which is probably what *"I just copy paste or drop the file."* means) does not support a text mode. Other utilities, such as FTP and git do. Hmm... re-check your dev server's documentation. Since there is support your sort of copying, there might be information about handling line endings in text files. You might have overlooked this earlier since you didn't know why it is an issue. — JaMiT, Nov 07 '21 at 02:13
@aysukaplan: If the data transfer program you are using does not support converting the text files as part of the transfer process, then the simplest solution in your case may be to use an advanced text editor which allows saving the file in both formats. According to [this link](https://stackoverflow.com/questions/8195839/choose-newline-character-in-notepad), Notepad++ is one such program and it is free and open source. — Andreas Wenzel, Nov 07 '21 at 02:17

score 0 · Accepted Answer · answered Nov 07 '21 at 02:10

The problem was caused by the text file not having UNIX-style line endings ("\n"), but instead having Windows-style line endings ("\r\n").

On Windows, when opening a file in text mode (without ios::binary), the "\r\n" line endings are automatically converted to "\n" line endings. On Linux, this conversion is not necessary, because they are supposed to be stored as "\n" in the first place. Therefore, on Linux, there is no difference between opening a file in text mode or binary mode, because in both cases, no conversion is performed.

However, uploading a text file with Windows-style line endings onto a Linux server is likely to cause trouble. As described above, on Linux, the "\r\n" line endings will not automatically be converted to "\n" line endings when opening the file in text mode, because on Linux, text files are expected to already have "\n" line endings. This is probably the reason why you say that getline reads an extra byte when running your program on the Linux server.

Most text editors running on Windows create Windows-style line endings. However, some advanced text editors also allow you to save the file with UNIX-style line endings. Also, some data transfer programs allow you to convert the files as part of the data transfer process.

In order to convert a text file from Windows-style line endings to UNIX-style line endings, you can use programs such as dos2unix or sed. You can also easily write such a program yourself, since the program does nothing else than remove all carriage return ('\r') characters:

#include <stdio.h>

int main(void)
{
    int c;

    while ( ( c = getchar() ) != EOF )
        if ( c != '\r' )
            putchar( c );
}

c++ fstream getline function adds \r to the line size

1 Answers1