Unable to convert a double/decimal value with 12 decimal places to a string using visual basic

Question

I am trying to convert a double with 12 decimal places to a string, but it only returns a string with 10 decimal places.

I have tried ToString, Format, FormatNumber, CStr and System.Convert.ToString. They all return 10 decimal places.

(Also same behavior with the decimal data type)

Example:

d = Date.Parse(sDate).ToOADate (value is 41261.001388888886)

d.ToString = 41261.0013888889

CStr(d) = 41261.0013888889

FormatNumber(d, 12) = 41,261.001388888900

In the case of Format and FormatNumber, when 12 places are specified, the value is rounded of to 10 digits.

This cropped up when I constructed a Filter for a DataView object.

Is there a way around this or is this a limitation in .Net?

A very similar question, with tons of information, is available here: [Round-twice error in .NET's Double.ToString method](http://stackoverflow.com/questions/11085052/round-twice-error-in-nets-double-tostring-method) — xfx, Jan 23 '13 at 07:00

Mark Hall · Answer 1 · 2013-01-23T08:09:47.010

2

In looking at the MSDN page for the Decimal Type it States:

You might need to use the D type character to assign a large value to a Decimal variable or constant. This requirement is because the compiler interprets a literal as Long unless a literal type character follows the literal, as the following example shows.

So based on your example

Module Module1

    Sub Main()
        Dim d As Decimal
        d = 41261.001388888886D

        Console.WriteLine(d)
        Console.WriteLine(CStr(d))
        Console.WriteLine(FormatNumber(d, 12))
        Console.WriteLine(d.ToString("#0.000000000000"))
        Console.ReadLine()
    End Sub

End Module

Displays:

41261.001388888886
41261.001388888886
4,1261.001388888886
41261.001388888886

Give this a try it seems to work correctly

Dim d As Decimal = CDec(String.Format("{0:G20}", Date.Parse(CStr(Now)).ToOADate))

edited Jan 23 '13 at 08:09

answered Jan 23 '13 at 06:55

Mark Hall

53,938
9
94
111

Thanks for the answer Mark, but I'm not sure how to apply that to my code as the value is being assigned by a Date function in VB. That is being assigned by d = Date.Parse(Now).ToOADate. – D. Saferite Jan 23 '13 at 07:06
I'm sure this may be hard to test and confirm but I don't think `ToOADate` is actually returning more than 10 decimals. – xfx Jan 23 '13 at 07:45
@xfx I can see that it is actually a double returning more than 10 decimals, just trying to get it to format correctly – Mark Hall Jan 23 '13 at 08:08
1

@D.Saferite Just added an additional example to my answer which seems to work. It is converting the Double to a String then converting to a Decimal. – Mark Hall Jan 23 '13 at 08:09

score 0 · Accepted Answer · answered Jan 23 '13 at 15:12

Double.ToString uses only 15 digit precision.

You should use d.ToString("R")

From MSDN:

The round-trip ("R") format specifier guarantees that a numeric value that is converted to a string will be parsed back into the same numeric value.

...

When a Single or Double value is formatted using this specifier, it is first tested using the general format, with 15 digits of precision for a Double and 7 digits of precision for a Single. If the value is successfully parsed back to the same numeric value, it is formatted using the general format specifier. If the value is not successfully parsed back to the same numeric value, it is formatted using 17 digits of precision for a Double and 9 digits of precision for a Single.

Unable to convert a double/decimal value with 12 decimal places to a string using visual basic

2 Answers2