vswprintf keeps prefixing a Byte Order Mark character

Question

I am still a rookie with C, and even newer to wide chars in C.

The below code should show

4 points to Smurfs

but it shows

4 points to Smurfs

In gdb I see this:

(gdb) p buffer
$1 = L" 4 points to  Smurfs",

But when I copy paste from the console, the spaces are magically gone:

(gdb) p buffer
$1 = L"4 points to Smurfs",

Also, buffer[0] contains this according to gdb: 65279 L' '

Apparently the character in question &#65279 is the Unicode Character 'ZERO WIDTH NO-BREAK SPACE' (U+FEFF). I retyped the code making sure I did not enter this. I don't know where this comes from. I also opened the code in notepad per https://stackoverflow.com/a/9691839/7602 and there is no extra chars there.

I wouldn't care if ncurses would stop showing this as a space.

Code (heavily cut down):

#include <time.h>
#include <stdio.h>
#include <errno.h>
#include <wchar.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <locale.h>
#define NCURSES_WIDECHAR 1
#include <ncursesw/ncurses.h>

#include "types.h"
#include "defines.h"
#include "externs.h"

WINDOW * term;

/*row column color n arguments */
void rccn(int row, int col, const wchar_t *fmt, ...)
{
    wchar_t buffer[80];
    int size;

    va_list args;
    va_start(args, fmt);
    size = vswprintf(buffer, 80, fmt, args);
    va_end( args );

    if(size >= 80){
        mvaddwstr(row, col, L"Possible hacker detected!");
    }else{
        mvaddwstr(row, col, buffer);
    }
}


int main(void)
{
    int ch;
    setlocale(LC_ALL,"");
    term = initscr();

    rccn(1,1,L"%i points to %ls",4,L"Smurfs");
    ch = getch();

    return EXIT_SUCCESS;
}

The problem goes 'away' with rccn(1,1,L"%i points to %ls",4,L"Smurfs"+1);

As if the wide encoding of the constant adds that char in front..

Perhaps change to `L"<%i> points to <%ls>"` to narrow the issue. — chux - Reinstate Monica, May 08 '18 at 15:12
If you run the code and redirect output to a file `a.out > t.bin`, what does a dump of the file show? — chux - Reinstate Monica, May 08 '18 at 15:17
It sounds like Windows doing something weird with byte order marks (U+FEFF is used as a byte order mark character). — Ian Abbott, May 08 '18 at 16:06
I actually run the code on Raspbian, just use Atom on Windows to do the coding with the ftp plugin. — tomdemuyt, May 08 '18 at 16:23

tomdemuyt · Answer 1 · 2018-05-09T12:32:13.853

0

Found it..

I had followed a tutorial where it was advised to add this compiler flag: -fwide-exec-charset=utf-32

My code was not running on Cygwin at all, and I read that Windows is utf-16 centered, so I removed that compiler flag and it started working on Cygwin.

Then out of curiosity I removed the compiler flag on Raspbian, and it is now working as expected there as well, no more byte order marks.

edited May 09 '18 at 12:32

answered May 09 '18 at 12:02

tomdemuyt

4,572
2
31
60

vswprintf keeps prefixing a Byte Order Mark character

1 Answers1