0

I have an object with a float property that I want to place into a text field. The problem is, that if I normally transform it with ".ToString()" it is written with scientific notation i. e. "1.5E+07" (And it cuts of all my decimal points). Because of that I wanted to transform it with the optional number parameter "#" and so I wrote ".ToString("0." + new string('#', 339)" to guarantee that it only cuts the useless zeros from my float, but writes all other numbers. No matter whether I use that or just "0.###" it always cuts all my decimal numbers. I. e. "-3740295.25" (float) turns into "-3740295" (string).

I really don't understand why this is happening and didn't find anything similar on the internet and most conversion are about doubles to strings anyway(which also doesn't work for me).

EDIT:

As requested here is a code snippet, which made me realize, if I create a completely fresh float it just works, I don't know why it doesn't work with the float from the object:

float testFloat1 = 1234.5544f;
float testFloat2 = (keyValuePair.Value as JsonFloat).Property; // -3740295.25
string testString1 = testFloat1.ToString("0.###"); // 1234.554
string testString2 = testFloat2.ToString("0.###"); // -3740295

This is the object in short:

public partial class JsonFloat : JsonValue
    {
        public float Property { get; set; }

        public JsonFloat(float property)
        {
            Property = property;
        }
    }

Kind Regards

ds600
  • 51
  • 9
  • 2
    Please post your code. Optimally a [mcve]. – Fildor Nov 04 '21 at 10:02
  • 5
    `float` has [7 decimal digits of precision](https://learn.microsoft.com/en-us/dotnet/api/system.single?view=net-5.0#floating-point-representation-and-precision). – GSerg Nov 04 '21 at 10:44
  • 1
    The issue is what @GSerg said. 3740295 is already eating up all the precision of the number. Try having `-37402956.25` (I added a 6 at the end of the whole number part) and you'll notice that even the 6 at the end there ends up as 0 and you get a rounding error up to 60 at the end. So basically, `float` does not have enough precision for your number. – Lasse V. Karlsen Nov 04 '21 at 12:49
  • 1
    @GSerg no, the precision of float is exactly 24 bits, which is roughly 6-7 decimal digits. There's no such thing as `9 decimal digits are maintained internally`. Using 9 digits won't allow full round trip for every value. See [Decimal Precision of Binary Floating-Point Numbers](https://www.exploringbinary.com/decimal-precision-of-binary-floating-point-numbers/) for more details – phuclv Nov 04 '21 at 13:00
  • @phuclv There is, if you click the documentation link above. It literally says, "*A Single value has up to 7 decimal digits of precision, although a maximum of 9 digits is maintained internally*". I'm probably not the one you meant to address though. – GSerg Nov 04 '21 at 13:05
  • @LasseV.Karlsen Amusingly, `Console.WriteLine(float.Parse("-3740295.25"));` gives `-3740295.2` in .NET 6rc2, even though the same .NET 6rc2 gives `-3740295` with `Console.WriteLine(float.Parse("-3740295.25").ToString("0.###"));`. Classic FW prints the whole number in both cases. – GSerg Nov 04 '21 at 13:08
  • @GSerg I must withdraw my previous statement, the precision of the float was indeed the error in my scenario, my thanks to both of you for the clarification, I'll keep this in mind for my future endeavours. If you could post that as an answer then I'll glady checkmark and upvote it – ds600 Nov 04 '21 at 13:16
  • @LasseV.Karlsen Again thank you for the clarification and example. I'll say the same to you as GSerg, if you post it as an answer I'll gladly checkmark and upvote it – ds600 Nov 04 '21 at 13:18
  • 1
    @GSerg: Yes, that documentation says “9 digits is maintained internally”, but [**it is wrong**](https://stackoverflow.com/a/61614323/298225). – Eric Postpischil Nov 04 '21 at 13:58
  • @GSerg it's wrong, float is a fixed precision type based on IEEE-754 single precision without an internal higher precision. Did you read my link above? – phuclv Nov 05 '21 at 02:14

2 Answers2

2

Main issue here is that float which maps to System.Single simply does not have enough precision.

And precision does not mean the digits after the decimal point, precision means the whole number.

Here, let me give you an example:

float f = 1234567890f;
Console.WriteLine(f.ToString("################"));

Output:

1234568000

As you can see, the number got "truncated" and rounded up, only about the 7 most significant digits are kept with a float.

This is also why it worked better with double, but this also have limited precision, just a high limit.

double d = 12345678901234567890d;
Console.WriteLine(d.ToString("##############################"));

Output:

12345678901234600000

As you can see here, double kept more of the digits but then gave up at about 15 significant digits.

You can see the limits specified in the documentation of System.Single and System.Double.

Even though System.Single says:

A Single value has up to 7 decimal digits of precision, although a maximum of 9 digits is maintained internally

(my emphasis)

those 9 digits might be temporary while the number is in flight during expression evaluation. The processor registers are slightly bigger and thus slightly more precise than the data types the values are ultimately stored into.

Lasse V. Karlsen
  • 380,855
  • 102
  • 628
  • 825
  • I still wonder what happens [here](https://dotnetfiddle.net/eMXYtw) though. Is it a bug in .NET 6? – GSerg Nov 06 '21 at 10:50
0

I don't know why, but if I place the float inside a double it just works. So this works:

double testFloat2 = (keyValuePair.Value as JsonFloat).Property; // -3740295.25
string testString2 = testFloat2.ToString("0.###"); // -3740295.25

If anyone knows why, then I would appreciate an explanation

ds600
  • 51
  • 9