25

I was fiddling around with parsing in C# and found that for every string I tried, string.StartsWith("\u2D2D") will return true. Why is that?

It seems it works with every char. Tried this code with .Net 4.5 the Debugger did not break.

for (char i = char.MinValue; i < char.MaxValue; i++)
{
    if(!i.ToString().StartsWith("\u2d2d"))
    {
        Debugger.Break();
    }
}
JLRishe
  • 99,490
  • 19
  • 131
  • 169
prydain
  • 365
  • 3
  • 11
  • here's a line of code to check a bunch. returns `false` for me: `Enumerable.Range(0, 10000).Any(x => !((char)x).ToString().StartsWith("\u2D2D"))` – DLeh Apr 23 '15 at 18:23
  • [Weird](https://ideone.com/q4kMsh). [Probably related](http://stackoverflow.com/questions/11467424/somestring-indexofsomestring-returns-1-instead-of-0-under-net-4/11467605#11467605). – CodeCaster Apr 23 '15 at 18:23
  • It must be because all strings are Georgian :D – Alex Apr 23 '15 at 18:24
  • @CodeCaster: https://ideone.com/0QffbT maybe related? – Caramiriel Apr 23 '15 at 18:24
  • @Caramiriel probably, but [`U+2D2D` is supposed to have a length of 1](http://www.fileformat.info/info/unicode/char/2d2d/index.htm). – CodeCaster Apr 23 '15 at 18:26
  • True. Ordinal culture seems to give false, so I'm looking into that now. – Caramiriel Apr 23 '15 at 18:27
  • Same for `2D2A` and the like. – Pierre-Luc Pineault Apr 23 '15 at 18:27
  • 2
    Might be useful to look at the reference source for `String.StartsWith` here: http://referencesource.microsoft.com/#mscorlib/system/string.cs,8281103e6f23cb5c – Asad Saeeduddin Apr 23 '15 at 18:38
  • 1
    U+2D2D was added in Unicode version 6.1 and it seems all 4 length characters of 6.1 have the same issue. From a few quick tests this seems the case for all versions > 6.0 – redrobot Apr 23 '15 at 19:01
  • Following the framework source, the comparison for non empty strings comes down to this line in `CompareInfo.cs`: http://referencesource.microsoft.com/#mscorlib/system/globalization/compareinfo.cs,603, and its all native from there on. I don't understand the win32 API sufficiently to be able to debug what is going on with the windows native code. – Asad Saeeduddin Apr 23 '15 at 19:09
  • A lot of the `UnicodeCategory.OtherNotAssigned` content matches this strange case. Can be checked with `CultureInfo.CurrentCulture.CompareInfo.Compare("", "" + (char)c, CompareOptions.None) == 0`, of which CurrentCulture = "en-US" – Caramiriel Apr 23 '15 at 19:20

1 Answers1

16

I think I'll have a try.

From what I get, is that U+2D2D was added in Unicode v6.1 (source / source).

The .NET framework, or the native calls rather, support a lower version:

The culture-sensitive sorting and casing rules used in string comparison depend on the version of the .NET Framework. In the .NET Framework 4.5 running on the Windows 8 operating system, sorting, casing, normalization, and Unicode character information conforms to the Unicode 6.0 standard. On other operating systems, it conforms to the Unicode 5.0 standard. (source)

Thus it is required to mark it as an ignorable character, which behaves just as if the character wasn't even there.

Character sets include ignorable characters, which are characters that are not considered when performing a linguistic or culture-sensitive comparison. (source)

Example:

var culture = new CultureInfo("en-US");
int result = culture.CompareInfo.Compare("", "\u2D2D", CompareOptions.None);
Assert.AreEqual(0, result);

string.StartsWith uses a similar implementation, but uses CompareInfo.IsPrefix(string, string, CompareOptions) instead.

Caramiriel
  • 7,029
  • 3
  • 30
  • 50
  • This makes more sense than the other answer, +1. – Der Kommissar Apr 23 '15 at 19:37
  • I just tested `.StartsWith()` with a blank string value, and it indicates true. I.e. `"abc".StartsWith("")` is *true*. Which makes sense, as it's basically saying that the underlying string starts with at least a *blank string*. This would indicate that when the underlying `Kernel32.dll` method `FindNLSStringEx` sees values it deems invalid, it simply strips them. (Or some other, lower-level method does.) The result is `string.StartsWith("")` when run. – Der Kommissar Apr 23 '15 at 19:49
  • @stakx: Thanks, but I don't mind if I don't. I was just wondering why this would happen in the first place, and maybe it would help me next time I face something like this myself. :) – Caramiriel Apr 23 '15 at 19:50
  • @EBrown: In the case of an empty prefix (`""`), `FindNLSStringEx` doesn't actually get called. The special case of 0-length prefixes is taken care of on the .NET side. That aside, according to this answer this it is the *logical* reason why `anyString.StartsWith("\u2D2D")` yields true. – stakx - no longer contributing Apr 23 '15 at 19:51
  • _I just tested .StartsWith() with a blank string value, and it indicates true._ Which is the defined (and logical) result. It does so if the string length == 0 – TaW Apr 24 '15 at 14:46