How to deal with the iOS detect the NSString language, English can not be detected accurately?

Question

Method：

- (NSString *)languageForString:(NSString *) text{
    return (__bridge NSString *)CFStringTokenizerCopyBestStringLanguage((CFStringRef)[text cStringUsingEncoding:NSUnicodeStringEncoding], CFRangeMake(0, MIN(text.length,100)));
}

Use：

NSLog(@"\"%@\" language is %@",@"Tokenizer",[self languageForString:@"tokenizer"]);
NSLog(@"\"%@\" language is %@",@"Tokenizer detect",[self languageForString:@"Tokenizer detect"]);
NSLog(@"\"%@\" language is %@",@"detect",[self languageForString:@"detect"]);
NSLog(@"\"%@\" language is %@",@"我们",[self languageForString:@"我们"]);
NSLog(@"\"%@\" language is %@",@"집안일",[self languageForString:@"집안일"]);
NSLog(@"\"%@\" language is %@",@"Démocratie",[self languageForString:@"Démocratie"]);
NSLog(@"\"%@\" language is %@",@"Tokenizer English",[self languageForString:@"Tokenizer English"]);
NSLog(@"\"%@\" language is %@",@"ここはデパートです",[self languageForString:@"ここはデパートです"]);

Output：

2013-01-09 16:12:28.582 TestCommandLine[6478:c07] "Tokenizer" language is tr<br/>
2013-01-09 16:12:28.586 TestCommandLine[6478:c07] "Tokenizer detect" language is tr<br/>
2013-01-09 16:12:28.586 TestCommandLine[6478:c07] "detect" language is cs<br/>
2013-01-09 16:12:28.587 TestCommandLine[6478:c07] "我们" language is zh-Hans<br/>
2013-01-09 16:12:28.560 TestCommandLine[6478:c07] "집안일" language is ko<br/>
2013-01-09 16:12:28.577 TestCommandLine[6478:c07] "Démocratie" language is fr<br/>
2013-01-09 16:12:28.590 TestCommandLine[6478:c07] "Tokenizer English" language is en<br/>
2013-01-09 16:12:28.591 TestCommandLine[6478:c07] "ここはデパートです" language is ja<br/>

How to become like：

2013-01-09 16:12:28.582 TestCommandLine[6478:c07] "Tokenizer" language is en<br/>
2013-01-09 16:12:28.586 TestCommandLine[6478:c07] "Tokenizer detect" language is en<br/>
2013-01-09 16:12:28.586 TestCommandLine[6478:c07] "detect" language is en<br/>
2013-01-09 16:12:28.587 TestCommandLine[6478:c07] "我们" language is zh-Hans<br/>
2013-01-09 16:12:28.560 TestCommandLine[6478:c07] "집안일" language is ko<br/>
2013-01-09 16:12:28.577 TestCommandLine[6478:c07] "Démocratie" language is fr<br/>
2013-01-09 16:12:28.590 TestCommandLine[6478:c07] "Tokenizer English" language is en<br/>
2013-01-09 16:12:28.591 TestCommandLine[6478:c07] "ここはデパートです" language is ja<br/>

From the docs for `CFStringTokenizerCopyBestStringLanguage `: *The result is not guaranteed to be accurate. Typically, the function requires 200-400 characters to reliably guess the language of a string.* It's going to be impossible to get good results with such short strings. — rmaddy, Jan 09 '13 at 08:30
This [question/answer](https://stackoverflow.com/questions/47890747/how-to-detect-text-string-language-in-ios/47890753#47890753) could be help. — Ahmad F, Dec 19 '17 at 17:03

score 1 · Answer 1 · edited Apr 16 '15 at 11:07

Here is my solution

- (NSString *)detectLanguage {

    if ([self isEmpty]) {
        return nil;
    }

    NSString *string = nil;

    // You can set a larger detect number here
    if (self.length > 30) {
        string = self;
    } else {
        NSMutableString *tempString = [NSMutableString stringWithString:self];

        while (tempString.length < 30) {
            [tempString appendFormat:@" %@",self];
        }

        string = tempString;
    }

    NSArray *tagschemes = [NSArray arrayWithObjects:NSLinguisticTagSchemeLanguage, nil];
    NSLinguisticTagger *tagger = [[NSLinguisticTagger alloc] initWithTagSchemes:tagschemes options:0];
    [tagger setString:string];
    NSString *language = [tagger tagAtIndex:0 scheme:NSLinguisticTagSchemeLanguage tokenRange:NULL sentenceRange:NULL];

    if (![language isEqualToString:@"und"]) {
        return language;
    }

    return (__bridge NSString *)CFStringTokenizerCopyBestStringLanguage((CFStringRef)string, CFRangeMake(0, MIN(string.length,400)));
}

score 0 · Answer 2 · answered Jan 09 '13 at 08:34

0

you can't recognize it that way .... at least not with ANY decent accuracy. you have to provide a MUCH longer string.

CFStringTokenizerCopyBestStringLanguage docs say it needs at least 200-400

-> there is no better way, we tried it with our own solution too and it needed more text for accuracy

answered Jan 09 '13 at 08:34

Daij-Djan

49,552
17
113
135

Detects the language of the word then be translated, not so long character – 八爪鱼 Jan 09 '13 at 08:52
sorry dont follow. the docs are clear and there simply is no better way with the SDK or with something else (though you may find a 3rd party component, SLIGHTLY better but in general they are equally 'bad') – Daij-Djan Jan 09 '13 at 09:28

How to deal with the iOS detect the NSString language, English can not be detected accurately?

2 Answers2