Recognizing tamil string and process them using c or c++ and the use of unicode

Question

The input is given in a language with a script other than the roman alphabets.A program in c or c++ must recognize them..

How do i take input in Tamil and split it into letters so that i can recognize each Tamil alphabet?

how do i use wchar_t and locale?

Answer 1

The C++ standard libraries do not handle Unicode completely, neither does C; you'd be better off using a library like Boost, which is cross platform

Answer 2

Including and using WinAPI and windows.h allow's you to use Unicode, but only on Win32 programs.

Answer 3

See here for a previous rant of mine on this subject.

Assuming that your platform is capable of handling Tamil characters, I suggest the following sequence of events:

I. Get the input string into a wide string:

#include <clocale>

int main()
{
  setlocale(LC_CTYPE, "");
  const char * s = getInputString(); // e.g. from the command line

  const size_t wl = mbstowcs(NULL, s, 0);
  wchar_t * ws = new wchar_t[wl];
  mbstowcs(ws, s, wl);
  //...

II. Convert the wide string into a string with definite encoding:

#include <iconv.h>

// ...

iconv_t cd = iconv_open("UTF32", "WCHAR_T");
size_t iin = wl;
size_t iout = 2 * wl; // random safety margin
uint32_t * us = new uint32_t[iout];
iconv(cd, reinterpret_cast<char*>(ws), &iin, reinterpret_cast<char*>(us), &iout);
iconv_close(cd);

// ...

Finally, you have in us an array of Unicode codepoints that made up your input text. You can now process this array, e.g. by looking each codepoint up in a list and checking whether it comes from the Tamil script, and do with it whatever you see fit.