22

My computer is configured with a culture that is not en-US.

When using the native Win32 GetDateFormat function, i get correctly formatted dates:

  • 22//11//2011 4::42::53 P̰̃M]

This is correct; and is also how Windows renders it:

  • the taskbar

    enter image description here

  • Region and Language settings

    enter image description here

  • Windows Explorer

    enter image description here

  • Outlook

    enter image description here

When i try to convert a date to a string in .NET using my current locale, e.g.:

DateTime.Now.ToString();
DateTime.Now.ToString(CultureInfo.CurrentCulture);

i get an incorrect date:

  • 22////11////2011 4::::42::::53 P̰̃M]

This bug in .NET is evident anyplace in Windows that uses the buggy .NET code:

  • Windows Event Viewer:

    enter image description here

  • Task Scheduler:

    enter image description here

  • SQL Server Management Studio:

    enter image description here

How do i make .NET not buggy?

How do i convert dates and times to strings using the current culture (correctly)?

Note: The user is allowed to set their Windows to any locale preferences they want. As it is now, my program will not handle valid settings properly. Telling the user, "Don't do that" is pretty mean-spirited.

A similar example comes from Delphi, which assumes that a date separator can never be more than one character. When Windows is configured with a locale that uses multiple characters for the date separator, e.g.:

  • sk-SK (Slovak - Slovakia) : .

where dates should be formatted as:

22. 11. 2011

the code library fails to accept a date separator longer than one character, and falls back to:

22/11/2011

In the past some might suggest that you not to bother with such edge cases. Such suggestions carry no weight with me.

i'll avoid getting into a pissing match with someone who wants to alter the meaning of my question by changing the title. But the question is not limited to pseudo-locales, specifically designed to find bugs in applications.

Bonus Chatter

Here's a unique list of date formats from around the world:

  • 11.11.25
  • 11.25.2011
  • 11/25/2011
  • 2011.11.25
  • 2011.11.25.
  • 2011/11/25
  • 2011-11-25
      1. 2011
  • 25.11.11
  • 25.11.2011
  • 25.11.2011 г.
  • 25.11.2011.
  • 25//11//2011
  • 25/11 2011
  • 25/11/2011
  • 25/11/2554
  • 25-11-11
  • 25-11-2011
  • 29/12/32

Of particular interest is the last example which doesn't use the gregorian calendar:

  • Arabic (Saudi Arabia) ar-SA: 29/12/32 02:03:07 م
  • Divehi (Maldives) dv-MV: 29/12/32 14:03:07
  • Dari/Pashto (Afghanistan) prf-AF / ps-AF: 29/12/32 2:03:07 غ.و

Although those are edge cases that you'd never have to worry about.


Update 14//12//2011:

Another demonstration of the bug is that Datetime.Parse cannot parse DateTime.ToString:

String s = DateTime.Today.ToString("d");   //returns "14////12////2011"
DateTime d = DateTime.Parse(s);            //expects "dd//MM//yyyy"

The .Parse throws an exception.


Update 02//8, 2012 09::56'12:

Any use of a date separator is depricated, in addition to being incorrect. From MSDN:

LOCALE_SDATE

Windows Vista and later: This constant is deprecated. Use LOCALE_SSHORTDATE instead. A custom locale might not have a single, uniform separator character. For example, a format such as "12/31, 2006" is valid.

LOCALE_STIME

Windows Vista and later: This constant is deprecated. Use LOCALE_STIMEFORMAT instead. A custom locale might not have a single, uniform separator character. For example, a format such as "03:56'23" is valid.

Community
  • 1
  • 1
Ian Boyd
  • 246,734
  • 253
  • 869
  • 1,219
  • The bug is the two extra slashes ? What is the name if this culture? – Magnus Nov 22 '11 at 22:01
  • 8
    How to make .NET not buggy? Submit a bug report to Microsoft! – Oded Nov 22 '11 at 22:05
  • @Magnus Yes, the locale defines a date separator (`LOCALE_SDATE`) of two forward slashes (e.g. `//`), and a date format (`SLOCALE_SSHORTDATE`) of `d//MM//yyyy`. dotNet is acting as though the separator is `////` with a format string of `d////MM////yyyy`. i know *why* there's a bug, it's taking each `/` and using it as its own internal "date separator token", replacing each `/` with `//` - which is the bug. – Ian Boyd Nov 22 '11 at 22:05
  • @Oded i cannot be the first person to hit this bug - therefore i must be doing it wrong. – Ian Boyd Nov 22 '11 at 22:08
  • 1
    @Ian - dunno. Looks like an edge case to me... Or something really strange going on with your computer. Have you been able to reproduce on other machines? – Oded Nov 22 '11 at 22:09
  • How have you built the culture? Did you use verbatim string literals? – Oded Nov 22 '11 at 22:11
  • @Oded It's supposed to be what the "correct" screenshots show. They're not supposed to be what the "incorrect" screenshots show. – Ian Boyd Nov 22 '11 at 22:12
  • @Oded No, it's the built-in locale `qps-ploc` (Pseudo-base). See linked so question for tutorial on how to change Windows to use it. – Ian Boyd Nov 22 '11 at 22:13
  • 3
    Stop looking for trouble and get back to work (you surely know what I mean) – Eduardo Molteni Nov 23 '11 at 00:41
  • 3
    @EduardoMolteni How do you convince developers to pay their "taxes?" (http://blogs.msdn.com/b/oldnewthing/archive/2005/08/22/454487.aspx) i assume you also don't honor the user's font preferences, dpi settings, honor when the user is running on battery, in a remote terminal services session, and you hard-code paths. No trouble to be found when you don't look for it. – Ian Boyd Nov 23 '11 at 14:22
  • @Oded Its fortunate that i used a built-in locale. Otherwise might say it's my own fault (*"Doctor, it hurts when i go like this." "Then don't go like that."*) – Ian Boyd Nov 23 '11 at 14:26
  • @EduardoMolteni Some date separators used in other cultures: `-`, `.`, `. `. i assume it's a bug in .NET, cause i'm a fairly bright guy, and i can see how they could easily make that mistake. On the other hand: perhaps .NET handles this correctly - but nobody outside the FCL team knows the proper method. – Ian Boyd Nov 23 '11 at 18:02
  • The "November" and "Tuesday" look like its displayed in the en-HACKER locale. – Omtara Nov 25 '11 at 17:54
  • Can you reproduce this culture in code so that we can reproduce the problem easily? In particular, I'd like to see how Noda Time deals with it... – Jon Skeet Nov 30 '11 at 14:09
  • @JonSkeet The first link details how to set Windows to the locale that trips up .NET. You can't simply use the control panel's **Region and Language** options, because the user interface doesn't let you specify the date separator. In theory you could reproduce the error by selecting a locale with a date separator that is not a slash (e.g. fr-ca uses `-`). Then use the date format of `M/d/yyyy`. .NET *should* show `11/30/2011`, but instead would show `11.30.2011` (because it interprets slash to mean "the DateSeparator character") – Ian Boyd Nov 30 '11 at 18:15
  • @IanBoyd: Yes, I could screw around with my Windows settings - but I'd rather not do that. A short but complete program which would have no lasting effect would be much nicer for anyone trying to reproduce this and work round it. If you *can't* reproduce it in code, that's worth knowing too. – Jon Skeet Nov 30 '11 at 18:36
  • @JonSkeet i see what you mean. i tried figuring out how to create a `CultureInfo` object where `CultureInfo.DateTimeFormat.ShortDatePattern = "M//d//yyyy"` and `CultureInfo.DateTimeFormat.DateSeparator = "//"`, but everything is read only. – Ian Boyd Nov 30 '11 at 19:19
  • @IanBoyd: You need to clone it first. Those patterns would *correctly* give "10////10////2011" - but the problem is that Windows shouldn't be populating the patterns like that in the first place. The ShortDatePattern should just be "M/d/yyyy" but with a DateSeparator of "//". – Jon Skeet Nov 30 '11 at 19:49
  • @JonSkeet Windows doesn't reserve `/` as a special **date separator replacement character**. Only `d`, `g`, `M`, `y` are reserved characters (http://msdn.microsoft.com/en-us/library/windows/desktop/dd317787(v=vs.85).aspx), everything else is copied to the output. If your format includes two `//`, then the output will contain two `//`. Look at `sk-sk` locale. The `LOCALE_SSHORTDATE` is `d. M. yyyy` (Note the *dot+space* separator). It is **not** `d/M/yyyy` with a date separator of `. `. If .net is looking for a `/` as a "date separator character", then it is making a mistake. – Ian Boyd Nov 30 '11 at 21:12
  • @IanBoyd: .NET format strings *do* use `/` to mean a date separator string (not necessarily a single character) and I don't believe that's a mistake at all. However, the conversion here *does* look like it's buggy. *Either* the date format should have a single "/" to represent "the date format" and use "//" as the separator itself, *or* it should use "//" in the format string but with a single "/" as the separator, *or* it should treat the "/" as a literal and escape it in the pattern. I can go into more detail if you'd like me to write it up as an answer. – Jon Skeet Nov 30 '11 at 21:15
  • @JonSkeet Problem is .NET is using the format strings from Windows, and is not adjusting them to become compatible with .NET's formatting system. The date format returned by Windows is correct, as it follows the Windows API rules. If .NET wants to use them, it must either follow the same rules, or adjust the format strings to become compatible with their formatting system. – Ian Boyd Nov 30 '11 at 21:37
  • @IanBoyd: Yes, that's what I've said: the conversion from the Windows settings to `DateTimeFormatInfo` properties looks like it's buggy. – Jon Skeet Nov 30 '11 at 21:38
  • @Magnus `qpc-ploc`, `qps-plocm`, `qps-ploca` (e.g. the problem doesn't happen with `en-US`) – Ian Boyd Dec 07 '11 at 15:19

2 Answers2

7

This specific bug is due to the transformation of some special characters that aren't escaped in the patterns like ShortDatePattern.

ShortDatePattern = "d//MM//yyyy";

/ in a pattern means "insert the date separator" but here the expansion is already done (at least on my system) when the string is copied from the system to the DateTimeFormat structure. Sadly it is missing an escaping (Obviously not visible on any language not using a special character as a separator and not visible in english as it is replaced with itself)

The only solution seem to be to escape the separators in all the patterns of the DateTimeFormat instance :

var c = new System.Globalization.CultureInfo("qps-ploc", true);
c.DateTimeFormat.ShortDatePattern =
        c.DateTimeFormat.ShortDatePattern.Replace("/", "'/'");
c.DateTimeFormat.LongTimePattern =
        c.DateTimeFormat.LongTimePattern.Replace(":", "':'");
Console.WriteLine(DateTime.Now.ToString(c));

Here's full code samples for all three common cases

Date to string

/// <summary>Convert a date to the short date string in the current locale (e.g. 30//11//2011)</summary>
/// <param name="value">A DateTime to be converted to a short date string</param>
/// <returns>A string containing the localized version of the date</returns>
public static String DateToStr(DateTime value)
{
    String format = CultureInfo.CurrentCulture.DateTimeFormat.ShortDatePattern;

    //The bug in .NET is that it assumes "/" in a date pattern means "the date separator"
    //What .NET doesn't realize is that the locale strings returned by Windows are the Windows format strings. 
    //The bug is exposed in locale's that use two slashes as for their date separator:
    //  dd//MM//yyyy
    // Which .NET misinterprets to give:
    //  30////11////2011
    // when really it should be taken literally to be:
    //  dd'//'MM'//'yyyy
    //which is what this fix does
    format = format.Replace("/", "'/'"); 

    return value.ToString(format);
}

Time to string

/// <summary>
/// Convert a time to string using the short time format in the current locale(e.g. 7::21 AM)
/// </summary>
/// <param name="value">A DateTime who's time portion will be converted to a localized string</param>
/// <returns>A string containing the localized version of the time</returns>
public static String TimeToStr(DateTime value)
{
    String format = CultureInfo.CurrentCulture.DateTimeFormat.ShortTimePattern;

    //The bug in .NET is that it assumes ":" in a time pattern means "the time separator"
    //What .NET doesn't realize is that the locale strings returned by Windows are the Windows format strings. 
    //The bug is exposed in locale's that use two colons as their time separator:
    //  h::mm::ss tt
    // Which .NET misinterprets to give:
    //  11::::39::::17 AM
    // when really it should be taken literally to be:
    //  h'::'mm'::'ss tt
    //which is what this fix does
    format = format.Replace(":", "':'"); 

    return value.ToString(format);
}

Datetime to string

/// <summary>
/// Convert a datetime to a string in the current locale (e.g. 30//11//2001 7::21 AM) 
/// </summary>
/// <param name="datetime">A DateTime to be converted to a general string in the current locale</param>
/// <returns>A string containing the localized version of the datetime</returns>
public static String DateTimeToStr(DateTime datetime)
{
    return DateToStr(datetime)+" "+TimeToStr(datetime);
}
Ian Boyd
  • 246,734
  • 253
  • 869
  • 1,219
Julien Roncaglia
  • 17,397
  • 4
  • 57
  • 75
  • Make me think that an even buggier case would be to use ':' as DATE separator and '/' as time separator, .Net will be even more confused... – Julien Roncaglia Nov 30 '11 at 14:13
  • @Graymatter's answer inspired me to realized that i wouldn't have to P/Invoke to the native API to get correct code. i came here to post, essentially, your answer, but you beat me to it. i'll append the actual static helper functions (which also have the benefit of not having to remember what format code gets you a date, a time, or a datetime. – Ian Boyd Nov 30 '11 at 16:43
  • @IanBoyd Instead of replacing the formating functions you could fix the CultureInfo structure and set `Thread.CurrentThread.CurrentCulture` with this all code will get the fix everywhere. – Julien Roncaglia Nov 30 '11 at 17:04
  • i would be concerned with that structure changing behind my back (i.e. the user changing their preferences while my application is running). i assume the .NET framework listens for `WM_SETTINGCHANGE` broadcasts and updates `Thread.CurrentThread.CurrentCulture` accordingly. Also it means background threads/delegates need to have the same fix applied. But yes, it would be a nice optimization to fix the culture info once, rather than every `DateToStr` / `TimeToStr` / `DateTimeToStr` – Ian Boyd Nov 30 '11 at 18:18
2

Your best bet is to log the bug with MS and then create an extension method that detects these edge cases and handles them.

Something like this (off the top of my head):

public static class DateTimeFix
{
    public static string FixedToString(this DateTime value)
    {
        if (IsEdgeCase())
            return FixEdgeCase(value);
        else
            return value.ToString();
    }

    // Edge case logic below
}

Then you use:

DateTime.Now.FixedToString()

in your code.

Graymatter
  • 6,529
  • 2
  • 30
  • 50
  • Your code idea made me realize that there was a purely managed fix (i was afraid i might have to PInvoke, when i can simply apply the fix to the format string that .NET should have). But VirtualBlackFox beat me to the code. – Ian Boyd Dec 07 '11 at 15:29