Weird character when displaying data from .dat file in c++

Question

I have an ipk.dat file containing student name and their GPA separated by semicolon. I'm trying to display the names of students who have a GPA greater than 3, but I get output with strange characters like this in the console.

Hidayat┬áSari            3.60 
Susila┬áBuana            3.27 
Krisna┬áSari             3.66 
Taufik┬áFatimah          3.38 
Bachtiar┬áDarma          3.70 
Yohanes┬áAnwar           3.93 
Harun┬áRatna             3.48 
Mega┬áZulfikar           3.32 
Zulfikar┬áAbdul          3.50 
Rahman┬áNirmala          3.37 
Amir┬áCinta              3.30 
Firdaus┬áLatifah         3.16 
Annisa┬áAli              3.65 
Eka┬áYuliana             3.14

This is my code:

#include <iostream>
#include <fstream>
#include <sstream>
using namespace std;

int main() {
    ifstream inGPA;
    string studentGPA;
    string studentName;
    inGPA.open("ipk.dat");

    if (inGPA.is_open()) {
        string line;
        while (getline(inGPA, line)) {
            stringstream ss(line);
            getline(ss, studentName, ';');
            getline(ss, studentGPA);

            if ( stod(studentGPA) >= 3.0) {
                cout << studentName << "     \t" << studentGPA << endl;
            }
        }
    }
    return 0;
}

And this is the inside of the ipk.dat file.The encoding for this file is UTF-8.

How do i fix this weird character issue?

Are the "spaces" in the input file actually "spaces" or some special character that only looks like a "space"? I would try to edit the dat file manually, remove the "space" between the names and explicitly add a space there to see if that would solve the issue — Pepijn Kramer, Nov 26 '21 at 06:11
@DedeKurniawan Can you copy/paste the text file in your question so that we can test it ourselves Instead of posting a picture of the text file. — Jason, Nov 26 '21 at 06:14
Also look at the input file in a hex-editor to see exactly what the "space" character is. — Some programmer dude, Nov 26 '21 at 06:14
@PepijnKramer ah i see. Thank you very much for the solution. It fixed. — Dede Kurniawan, Nov 26 '21 at 06:15
The spaces are UTF-8 encoded U+00A0, also known as non-breaking space. — Raymond Chen, Nov 26 '21 at 06:23

score 0 · Accepted Answer · answered Nov 26 '21 at 10:01

The non-breaking space might be unwanted input, but if you have names with non-ASCII characters, you will have the same problem.

Part of the problem here is that your terminal doesn't know that you are sending UTF-8 encoded characters.

If you are on Windows you can refer to this question.
The basic idea is to set the terminal to understand UTF-8 first:

#include <Windows.h>

int main() {
    SetConsoleOutputCP(CP_UTF8); // set output to UTF-8
    // your code
}

This will print your non-breaking space characters normally.

Note: This change doesn't only last for the execution of your program.
If you run your unfixed program after your fixed one, the program will seemingly work, until you run it in a fresh terminal.

Jason · Answer 2 · 2021-11-26T08:45:44.833

Since you have already fixed your issue(which seem to with the input file rather than with the program), i want to propose one modification to ͟r͟e͟m͟o͟v͟e͟ ͟t͟h͟e͟ ͟r͟e͟d͟u͟n͟d͟a͟n͟c͟i͟e͟s͟. In particular, you don't need

//no need for these three statements
stringstream ss(line);
getline(ss, studentName, ';');
getline(ss, studentGPA);

in your program. Instead you can just directly use getline on studentName and studentGPA as shown below.

#include <iostream>
#include <fstream>
#include <sstream>
using namespace std;

int main() {
    ifstream inGPA;
    string studentGPA;
    string studentName;
    inGPA.open("input.txt");

    if (inGPA.is_open()) {
        string line;
        //just getline directly into variables studentName and studentGPA
        while (getline(inGPA, studentName, ';'), getline(inGPA, studentGPA)) {

            if ( stod(studentGPA) >= 3.0) {
                cout << studentName << "     \t" << studentGPA << endl;
            }
        }
    }
    return 0;
}

Weird character when displaying data from .dat file in c++

2 Answers2