-3

How can I limit a float to a defined amount of digits? Let's say I want a precision of 4, then it should be:

14.49193 -> 14.49
1.449193 -> 1.449
0.449193 -> 0.4492
0.000449193 -> 0.0004492
SalkinD
  • 753
  • 9
  • 23
  • Is this for display purposes only or the actual value you want in the variable? – jonsca Jun 11 '22 at 08:57
  • Both. Remove irrelevant imprecision of float operations. – SalkinD Jun 11 '22 at 08:58
  • 1
    https://stackoverflow.com/questions/48212254/set-the-precision-for-decimal-numbers-in-c-sharp – Behnam Asaei Jun 11 '22 at 08:59
  • 2
    you can't. That's not how the `float` data type works. – bolov Jun 11 '22 at 09:00
  • @Gserg The answers to this question are unfortunately not correct. (try with <0.1 or only 4 digits). I understand mathematically why this does not work, but am not good enough to modify it correctly... – SalkinD Jun 11 '22 at 09:17
  • 1
    @SalkinD True. Replace `Math.Log10(x)` with `Math.Max(Math.Log10(x), -1)` - which will then not count the significant leading zero like in your 0.449193 -> 0.4492. Otherwise use `Math.Max(Math.Log10(x), 0)`, then it will be 0.449193 -> 0.449 like I would expect for four digit precision. – GSerg Jun 11 '22 at 09:28
  • Thanks @Gserg, I somehow figured out a method that does what I want. Posted it as answer. – SalkinD Jun 11 '22 at 10:31

1 Answers1

0

I resolved it with this method, there may be a smoother solution.

public static float LimitDigits(float val,int digits) {
    if (val == 0) return 0;
    return (float)((val<1.0)?Math.Round(val,digits+(int)Math.Floor(-(Math.Log10(val)))):Math.Round(val, Math.Max(0,digits-(int)Math.Floor((Math.Log10(val)+1)))));
}
SalkinD
  • 753
  • 9
  • 23
  • 1
    `Math.Max(Math.Log10(x), 0)` works for both greater and smaller than 1.0. Your entire formula is `Math.Round(val, digits - (int)Math.Floor(Math.Max(Math.Log10(x), -1) + 1));` – GSerg Jun 11 '22 at 10:39
  • Yes, but under 1.0 Log(x) is negative, so <1.0 its digits+result and over 1.0 its digits-result – SalkinD Jun 11 '22 at 10:43
  • 1
    That is why you have `Math.Max`. Don't need to over complicate it. See https://dotnetfiddle.net/cMFNrq – GSerg Jun 11 '22 at 10:44
  • https://dotnetfiddle.net/10Ec59 does not work with low numbers :) – SalkinD Jun 11 '22 at 10:46
  • Looks correct to me? `LimitDigits` doesn't work, `LimitDigits2` does, which is the one I'm suggesting. – GSerg Jun 11 '22 at 10:55
  • LimitDigits gives 0.000449193 -> 0.0004492 so it shows 4 relevant digits. LimitDigits2 outputs 0.000449193 -> 0.0004 , the last 3 relevant digits are missing. – SalkinD Jun 11 '22 at 10:57
  • 1
    Then we have a completely different understanding of relevant digits. All digits in `0.0004` are relevant. – GSerg Jun 11 '22 at 10:58
  • Hm, difficult to explain then, true that. I want to show the "rightmost" 4 digits of a number, including preceding zeroes. Don't know how to call that correctly... (It's laboratory values, as the lab has never a precision more than 4 but has values from 1e-6 to 1.000.000; as I had to do a unit conversion float imprecision added senseless micro values like 0.39999998 that I need to get back to 0.4) – SalkinD Jun 11 '22 at 11:04
  • Then maybe you want the [scientific format](https://stackoverflow.com/a/30975539/11683) in the first place? `val.ToString("0.000E0");`. – GSerg Jun 11 '22 at 11:20
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/245514/discussion-between-salkind-and-gserg). – SalkinD Jun 11 '22 at 11:27