Reading Files in from UTF8

Question

I am trying to reading in a file that contains characters in UTF-8 format.

    FILE * f = fopen(argv[1], "r");
    if(f == NULL){
            printf("cannot open %s\n", argv[1]);
            exit(-1);
        }
    unsigned int c = getc();
    while(c != EOF){
        printf("%d\n", c);  // UB
        c = getchar();

    }

How do I read the files in so that they are in bit representation? For example that it would look like: 0xA3. Right now it's printing actual integers. Basically, how do I read in characters in bit wise format?

It's printing integers because of the way you've written the code. Can you explain more what you're trying to do? You do know UTF-8 is a multibyte encoding right? — Retired Ninja, Jan 31 '17 at 13:18
All data in modern binary computers are in binary form. It's just a matter of how you choose to *display* that data. In your case I suggest you read more about [`printf`](http://en.cppreference.com/w/c/io/fprintf) and its different formats. Also, just about *any* [beginners book](http://stackoverflow.com/questions/562303/the-definitive-c-book-guide-and-list) or tutorial would have told you how to print any integer in hexadecimal notation. — Some programmer dude, Jan 31 '17 at 13:19
@RetiredNinja basically my goal it to decode UTF8 to utf32 ... — Joe Caraccio, Jan 31 '17 at 13:27
utf-8 is multi byte encoding character set. so you have to use wide char functions to operate on file .... — rajesh6115, Jan 31 '17 at 13:49
The code as posted doesn't need utf-32 for what it attempts to do. — n. m. could be an AI, Jan 31 '17 at 14:00
Do you want hexadecimal output like `0xA3` or "bit wise format" like `10100011`? — chux - Reinstate Monica, Jan 31 '17 at 15:19
Do you want output a byte at a time or a UTF8 code point at a time? — chux - Reinstate Monica, Jan 31 '17 at 15:20
It isn't clear what is meant by "bit wise format" or "bit representation". Show a sample input file and desired output. — n. m. could be an AI, Jan 31 '17 at 15:27

rajesh6115 · Answer 1 · 2017-01-31T15:42:12.660

2

sample Example for utf-8 file reading.....

#include <stdio.h>
#include <wchar.h>
#include <locale.h>
#include <stdlib.h>

int main(int argc, char *argv[]){
        setlocale(LC_ALL, "en_US.UTF-8");
        FILE * f = fopen(argv[1], "r");
        if(f == NULL){
                printf("cannot open %s\n", argv[1]);
                exit(-1);
        }
        wchar_t wc;
        while((wc=fgetwc(f))!=WEOF){
                //wprintf(L"%lc", wc);
                // for output as hex of uthf-8 characters
                wprintf(L"0X%X,",(wint_t) wc);
        }
        wprintf(L"\n");
        fclose(f);
        return 0;
}

edited Jan 31 '17 at 15:42

answered Jan 31 '17 at 13:50

rajesh6115

705
9
21

1

This is not portable. – n. m. could be an AI Jan 31 '17 at 13:56
I think all functions used here are available in standard c. I am not getting for what it is not portable? can you en-light us . – rajesh6115 Jan 31 '17 at 14:10
3

The locale name is not portable. It is not guaranteed that an implementation supports utf-8 locales. – n. m. could be an AI Jan 31 '17 at 14:16
1

Code should check the result of `setlocale(LC_ALL, "en_US.UTF-8")` for success and handle that to be portable. – chux - Reinstate Monica Jan 31 '17 at 15:15
3

BTW: OP wants output like `0xA3` which certainly will not result from `wprintf(L"%lc", wc)`. – chux - Reinstate Monica Jan 31 '17 at 15:17
@chux Portable code should work (do what it is designed to do) on a number of platforms. Printing "sorry, no such locale" isn't useful work. – n. m. could be an AI Jan 31 '17 at 15:22
@chux What OP wants is not exactly clear. Do you know what is meant by "bit representation" or "bit wise format"? – n. m. could be an AI Jan 31 '17 at 15:25
@n.m. Disagree about usefulness of code stopping with a message as to why it can not complete. If the platform does not natively handle UTF8, rollings ones own solution is not trivial, but true could be done. OTOH other locales than `"en_US.UTF-8"` may handle UTF8. – chux - Reinstate Monica Jan 31 '17 at 15:28
@n.m. Agree OP's post lacks clarity and so I [asked OP](http://stackoverflow.com/questions/41958597/reading-files-in-from-utf8/41959270?noredirect=1#comment71100955_41958597) to about that as you have too. – chux - Reinstate Monica Jan 31 '17 at 15:29
@chux Stopping with an error message could be useful at times, but a message about inability to do useful work isn't the same as work. – n. m. could be an AI Jan 31 '17 at 15:37

Reading Files in from UTF8

1 Answers1