C programming, unicode/UTF-8 special characters not showing up on linux console

Question

I am a C beginner and trying this and that.

Generally, I want to use special charaters "blocks" to have some kind of percentage display.

one is a kind of pie chart on a 3x3 char matrix, one is just going upwards filling a char from the bottom to the top, and eventually a string of two chars shall from left to right display the grow to 100 percent.

Pricipally it seems to work out, but obviously my console has some problems to display the output. non correctly displayed chars seem to lead stdout not to come to \n.

Some research make me believe that the locale settings of the terminal (bash on a debian derivat) have some problem to show the chars, while Code::Blocks has no problems to show the chars and some of the chars are even shown correctly. sorry, also I am some confused about unicode and utf-8 here.

on the screenshow below you can see some code displayed, my locale settings, and some output of the code.

How can I solve the problem?

BTW: I would like to stick to printf if possible to keep things easy for me.

#include <stdio.h>
#include <stdlib.h>
#include <locale.h>

// some little helpers
# define hideCursor() printf("\e[?25l")
# define showCursor() printf("\e[?25h")
# define clear() printf("\033[H\033[J")

// # define WIDE_ORIENTED 1

int msleep(long tms);

int main()
{
setlocale(LC_ALL, "");
// fwide( stdout, WIDE_ORIENTED );

char percentage[21][4][4] = {{"   ","   ","   ","  0"},
                             {" ▐ ","   ","   ","  5"},
                             {" ▐▌","   ","   "," 10"},
                             {" ▐▛","   ","   "," 15"},
                             {" ▐█","   ","   "," 20"},
                             {" ▐█","  ▀","   "," 25"},
                             {" ▐█","  █","   "," 30"},
                             {" ▐█","  █","  ▀"," 35"},
                             {" ▐█","  █","  ▜"," 40"},
                             {" ▐█","  █","  █"," 45"},
                             {" ▐█","  █"," ▐█"," 50"},
                             {" ▐█","  █"," ██"," 55"},
                             {" ▐█","  █","▐██"," 60"},
                             {" ▐█","  █","▟██"," 65"},
                             {" ▐█","  █","███"," 70"},
                             {" ▐█","▄ █","███"," 75"},
                             {" ▐█","█ █","███"," 80"},
                             {"▄▐█","█ █","███"," 85"},
                             {"▙▐█","█ █","███"," 90"},
                             {"█▐█","█ █","███"," 95"},
                             {"███","█ █","███","100"}};

char percentageUpwards[10] = {" ▁▂▃▄▅▆▇██"};

char percentageSidewards[20][2] = {"  ","▏ ","▎ ","▍ ","▌ ","▌ ","▋ ","▊ ","█ ","▉ ","▉▏","▉▎","▉▍","▉▌","▉▌","▉▋","▉▊","▉█","▉▉","▉▉"};


// does already nor work :-/
    clear();
    int counter = 0;
    for (int i = 0 ; i <= 100 ; i++) {
    //printf("%d %d\n", (i), (i % 5));
        if ((i % 5) == 0) {
            clear();
            printf("Squared Percentage:\n");
            printf("%s\n",percentage[counter][0]);
            printf("%s\n",percentage[counter][1]);
            printf("%s\n",percentage[counter][2]);
            printf("%s\n",percentage[counter][3]);
            msleep(500); // nappy for easier following
            counter++;
        }
    }

// only a percentage display going upwards
    counter = 0;
    for (int i = 0 ; i <= 100 ; i++) {
        if ((i % 11) == 0) {
            clear();
            printf("Upwards:\n");
            printf("%c\n", percentageUpwards[counter]);
            fflush(stdout);
            printf("%d\n", i);
            msleep(500); // nappy for easier following
            counter++;
        }
    }

// only a percentage display going upwards
    counter = 0;
    for (int i = 0 ; i <= 100 ; i++) {
        if ((i % 5) == 0) {
            clear();
            printf("Sidewards:\n");
            printf("%s\n", percentageSidewards[counter]);
            fflush(stdout);
            printf("%d\n", i);
            msleep(500); // nappy for easier following
            counter++;
        }
    }

// Test only
printf("%s\n", percentage[15][0]);
printf("%s\n", percentage[15][1]);
printf("%s\n", percentage[15][2]);
printf("%s\n", percentage[15][3]);
printf("\n");
printf("%c\n", percentageUpwards[5]);
printf("\n");
printf("%s\n", percentageSidewards[10]);

    return 0;
}

// have a nap in 1/1000th of a second steps
int msleep(long tms)
{
    struct timespec ts;
    int ret;
    if (tms < 0) {
        return -1;
    }
    ts.tv_sec = tms / 1000;
    ts.tv_nsec = (tms % 1000) * 1000000;
    do {
        ret = nanosleep(&ts, &ts);
    }
    while (ret);
    return ret;
}

a simple screenie

Check UTF8 enabled in your terminal. https://stackoverflow.com/questions/5306153/how-to-get-terminals-character-encoding — Florent, Nov 29 '20 at 14:05
I suspect that your `percentage` array needs more space in the third dimension — change the second 4 to at least 5. I don't recall the Unicode code points for the block characters you are using, but each one requires at least two and possibly three bytes in UTF-8. — Jonathan Leffler, Nov 29 '20 at 14:05
Perhaps using L" " syntax for litteral string ? And wchar_t for char. — Florent, Nov 29 '20 at 14:36
@Florent Thanks, I double-checked, but it is utf-8. (see also the screenshot.) Anyway, I think you are right and this is (besides wrong dimensions for my arrays) the problem. Your solution to use litteral strings and wchar_t I read also about, but seems to be some to far advanced for my humble beginnings :-) — jagottsicher, Nov 30 '20 at 03:38

score 0 · Answer 1 · answered Nov 29 '20 at 17:54

As I guessed in a comment, the major problem is that your code tries to store too much data in arrays that are too small.

When I copy your code, it doesn't compile because the strings initializing the array are too big. The characters you're using seem to be in the range U+2580..U+259F, which means that they require 3 bytes when represented in UTF-8 (U+2580 = 0xE2 0x96 0x80 = U+2580; 0xE2 0x96 0x9F = U+259F).

You have to increase the size of your arrays — C counts bytes, not characters when using char, signed char or unsigned char. Using 16 is overkill; 10 would be sufficient; likewise 8 could be just 7. You also need to treat the percentageUpwards array as an array of strings of at least size 4. Putting those changes together yields this code, which more or less works on both a Mac running (sigh) macOS 10.14.6 Mojave and a Linux box running RHEL Server release 7.4 (Maipo) — the latter running over ssh into a Terminal on the Mac.

#include <stdio.h>
#include <stdlib.h>
#include <locale.h>
#include <time.h>

// some little helpers
# define hideCursor() printf("\e[?25l")
# define showCursor() printf("\e[?25h")
# define clear() printf("\033[H\033[J")

// # define WIDE_ORIENTED 1

int msleep(long tms);

int main(void)
{
    setlocale(LC_ALL, "");
// fwide( stdout, WIDE_ORIENTED );

    char percentage[21][4][16] =
    {
        { "   ", "   ", "   ", "  0" },
        { " ▐ ", "   ", "   ", "  5" },
        { " ▐▌", "   ", "   ", " 10" },
        { " ▐▛", "   ", "   ", " 15" },
        { " ▐█", "   ", "   ", " 20" },
        { " ▐█", "  ▀", "   ", " 25" },
        { " ▐█", "  █", "   ", " 30" },
        { " ▐█", "  █", "  ▀", " 35" },
        { " ▐█", "  █", "  ▜", " 40" },
        { " ▐█", "  █", "  █", " 45" },
        { " ▐█", "  █", " ▐█", " 50" },
        { " ▐█", "  █", " ██", " 55" },
        { " ▐█", "  █", "▐██", " 60" },
        { " ▐█", "  █", "▟██", " 65" },
        { " ▐█", "  █", "███", " 70" },
        { " ▐█", "▄ █", "███", " 75" },
        { " ▐█", "█ █", "███", " 80" },
        { "▄▐█", "█ █", "███", " 85" },
        { "▙▐█", "█ █", "███", " 90" },
        { "█▐█", "█ █", "███", " 95" },
        { "███", "█ █", "███", "100" },
    };

    char percentageUpwards[][4] =
    {
        " ", "▁", "▂", "▃", "▄", "▅", "▆", "▇", "█", "█", 
    };

    char percentageSidewards[20][8] =
    {
        "  ", "▏ ", "▎ ", "▍ ", "▌ ", "▌ ", "▋ ", "▊ ", "█ ", "▉ ",
        "▉▏", "▉▎", "▉▍", "▉▌", "▉▌", "▉▋", "▉▊", "▉█", "▉▉", "▉▉",
    };

    // does already not work :-/
    clear();
    int counter = 0;
    for (int i = 0; i <= 100; i++)
    {
        // printf("%d %d\n", (i), (i % 5));
        if ((i % 5) == 0)
        {
            clear();
            printf("Squared Percentage:\n");
            printf("%s\n", percentage[counter][0]);
            printf("%s\n", percentage[counter][1]);
            printf("%s\n", percentage[counter][2]);
            printf("%s\n", percentage[counter][3]);
            msleep(500); // nappy for easier following
            counter++;
        }
    }

    // only a percentage display going upwards
    counter = 0;
    for (int i = 0; i <= 100; i++)
    {
        if ((i % 11) == 0)
        {
            clear();
            printf("Upwards:\n");
            printf("%s\n", percentageUpwards[counter]);
            fflush(stdout);
            printf("%d\n", i);
            msleep(500); // nappy for easier following
            counter++;
        }
    }

    // only a percentage display going upwards
    counter = 0;
    for (int i = 0; i <= 100; i++)
    {
        if ((i % 5) == 0)
        {
            clear();
            printf("Sidewards:\n");
            printf("%s\n", percentageSidewards[counter]);
            fflush(stdout);
            printf("%d\n", i);
            msleep(500); // nappy for easier following
            counter++;
        }
    }

    // Test only
    printf("%s\n", percentage[15][0]);
    printf("%s\n", percentage[15][1]);
    printf("%s\n", percentage[15][2]);
    printf("%s\n", percentage[15][3]);
    printf("\n");
    printf("%s\n", percentageUpwards[5]);
    printf("\n");
    printf("%s\n", percentageSidewards[10]);

    return 0;
}

// have a nap in 1/1000th of a second steps
int msleep(long tms)
{
    struct timespec ts;
    int ret;
    if (tms < 0)
    {
        return -1;
    }
    ts.tv_sec = tms / 1000;
    ts.tv_nsec = (tms % 1000) * 1000000;
    do
    {
        ret = nanosleep(&ts, &ts);
    }while (ret);
    return ret;
}

I can't sensibly show the output; it is dynamic and takes about 26 seconds to execute because of the msleep() calls.

Thanks, this helped a lot already. I understand now that my arrays need to address the size not in char but for storage needed bytes. Your code runs out of the box on my debain derivate and shows already correct percentages counting up. So in the background and from a logical and syntactical point of view it is fine. Unfortunately, there are partly still some dummy charcters shown in output, which riun the effect of the "animations of chars, respectively, strings". — jagottsicher, Nov 30 '20 at 03:28

C programming, unicode/UTF-8 special characters not showing up on linux console

1 Answers1