Splitting unicode string with combined/dependent characters together

Question

Suppose i have a bangla string like "সাজানো". I need to split that into "সা" "জা" "নো". I have tried the ToCharArray() method but yes it splits the following string into 'স','া','জ','া','ন', 'ো'.

So the problem is that i want to split a string to another string array with combined/dependent characters together.Like "সা" should be separated from "সাজানো", not in individual char like 'স'and 'া'.

You cannot split Indian languages like that. Check this SO post [Converting Unicode string to unicode chars in c# for indian languages](http://stackoverflow.com/questions/13966487/converting-unicode-string-to-unicode-chars-in-c-sharp-for-indian-languages) for a solution — bansi, Sep 15 '14 at 05:27

score 0 · Answer 1 · edited Jun 10 '15 at 04:42

0

try this:

string[] SplitString(string S)
{
    return S.Split('া');
}

private void button1_Click(object sender, EventArgs e)
{
    string B = "সাজানো";
    var vv = SplitString(B);
    foreach (var item in vv)
    {
        MessageBox.Show(item.ToString());
    }
}

edited Jun 10 '15 at 04:42

Nic

12,220
20
77
105

answered Sep 15 '14 at 05:19

Mehdi Khademloo

2,754
2
20
40

this won't work, what happens the vowel part is `ो`, `ि` ... etc? – bansi Sep 15 '14 at 05:35
@bansi we don't know the Bangla. But it work with the instance of question _সাজানো_ => _"সা" "জা" "নো"_ – Mehdi Khademloo Sep 15 '14 at 06:38

score 0 · Answer 2 · edited Jun 10 '15 at 04:41

This works too:

string text = "সাজানো";
var textCharArray = text.ToCharArray();
var tokens = new List<string>();

for (int i = 0; i < textCharArray.Length; i++)
{
    char c = textCharArray[i];
    if (char.GetUnicodeCategory(c) == System.Globalization.UnicodeCategory.SpacingCombiningMark)
    {
        string token = $"{tokens.Last()}{c}";
        tokens.RemoveAt(tokens.Count() - 1);
        tokens.Add(token);
    }
    else
    {
        tokens.Add($"{c}");
    }
}

foreach (string token in tokens) 
    Console.WriteLine(token);

The code is kind of sloppy. Its late.

You can also test for Surrogates as well if you're working with those.

Splitting unicode string with combined/dependent characters together

2 Answers2