3

I'm using Qt5 along with Visual Studio 2012 and recently wrote a logger class, which basically redirect string streams to the file. The other day I realised that there is no "special" characters support (eg. polish, german, russian).

qDebug() << "Special characters: ąężźćłóĄĘŻĆŁÓ";

Is producing the following output:

Special characters: �꿟����ʯƣ�

I have tried multiple Unicode settings listen in File -> Advanced Save Options.

enter image description here

However, there is no option to save the file without the BOM signature and I think that might be the issue, since when I change the file encoding through the Notepad++ to UTF-8 (without BOM), then compile, everything is working fine... unfortunately until I make any changes in the Visual Studio.

I have also tried setting compiler encoding to Unicode:

enter image description here

Is there any solution for Visual Studio to change the encoding to UTF-8 without BOM signature?

Code snippet which writes to file:

file = new QFile;
file->setFileName(fileName);
file->open(QIODevice::Append | QIODevice::Text);

[..]

QTextStream out(file);
out.setCodec("UTF-8");
out << QDateTime::currentDateTime().toString("dd.MM.yyyy hh:mm:ss ") << value << "\n";

I've been also trying using value.toUtf8().

Lucas
  • 3,517
  • 13
  • 46
  • 75
  • Except for the BOM if you view the file in a Hex viewer does it look the same when saved as UTF-8 from VS-2012 and Notepad++? – Richard Critten Oct 13 '16 at 20:28
  • @RichardCritten No. Btw. VS-2012 UTF-8 encoding is detected in Notepad++ as `UCS-2 LE (Little Endian) BOM`. – Lucas Oct 13 '16 at 20:33
  • Is this going to console window output? Native windows output should be okay. Convert to UTF-16 and show with MessageBox. By the way, which Windows version are you using? Vista,7,8,10...? – Barmak Shemirani Oct 13 '16 at 20:47
  • @BarmakShemirani Yes, it's being printed in the console output (only on debug builds) and also it's being written to the log file, although I don't really care about the console output since as I said, I'm only relying on the file output here. – Lucas Oct 13 '16 at 20:54
  • Lets forget about console output for now. Show the code which writes to file (probably writes in UTF-8), and show the code which reads the UTF-8 file. Note that Windows uses UTF-16, so a native Windows program will have trouble reading Qt program without the correct conversion. – Barmak Shemirani Oct 13 '16 at 21:01
  • Also try adding `u8` prefix: `u8"Special characters: ąężźćłóĄĘŻĆŁÓ"` – Barmak Shemirani Oct 13 '16 at 21:08
  • @BarmakShemirani Code snippet has been added to the question. I'm only writing to the file, no reading. I'm simply opening it in Notepad++ or Windows native notepad. – Lucas Oct 13 '16 at 21:35
  • @BarmakShemirani Regarding the `u8` prefix - it's producing an error. `http://i.imgur.com/5P8hA7m.png` `(expected a ';')` – Lucas Oct 13 '16 at 21:38
  • Try wrapping your string literal in the [QStringLiteral](http://doc.qt.io/qt-5/qstring.html#QStringLiteral) macro. – MrEricSir Oct 13 '16 at 21:49
  • @MrEricSir Well, surprisingly it's working but generating unnecessary quotation marks. – Lucas Oct 13 '16 at 21:57
  • @Lucas That's the expected behavior of qDebug. See this for more info: http://stackoverflow.com/questions/27976581/why-is-qstring-printed-with-quotation-marks – MrEricSir Oct 13 '16 at 21:59
  • In Visual Studio leave the encoding at `"UTF-8 codepage 65001"` – Barmak Shemirani Oct 13 '16 at 22:04
  • @MrEricSir Alright I see. You can post an answer concerning on what `QStringLiteral` have changed for string to be working with UTF8 encoding. – Lucas Oct 13 '16 at 22:05
  • @MrEricSir Btw. is this possible to pass kind of a variable to `QStringLiteral`? I'd like to use it in the function so I wouldn't have to wrap everything with `QStringLiteral`. – Lucas Oct 13 '16 at 22:11
  • @Lucas Of course not -- the phrase "string literal" is right there in the name of the macro. – MrEricSir Oct 13 '16 at 22:52

1 Answers1

1

After many unsuccessful tries, I have two possibilities to fix the encoding issue:

  1. Plugin for Visual Studio: https://vlasovstudio.com/fix-file-encoding/, this plugin prevents Visual Studio from adding BOM to the beginning of the file, so that way all of my files can have UTF-8 encoding and raw strings can contain special character and they will be displayed/written without any issues,

  2. This one is suggested by @MrEricSir in the comments. The idea is to use QStringLiteral function on string containing special characters.

    Like this: QStringLiteral("ąśżęłóĄŚŻĘŁÓ");

I personally picked the first method since it don't force me to keep using additional functions everytime I'd like to print special characters. In both cases, results are the same.

Thanks for everyone who posted a comment and tried to help.

Lucas
  • 3,517
  • 13
  • 46
  • 75