1

This is a bit of a brain teaser for me. We use a remote server to store our data. I have no control over naming convention or how data is stored in there. There is an sproc we have written on that server to call values from a table there. One of the values, price, comes back as 6.52....for this particular example.

enter image description here

The problem was when I then called that value after it was stored in our own local database it showed as 6.51.

Thinking it was something to do with rounding, and that it wasn't happening as the data transitioned through my c# program, I pulled a datatable that only retrieved the row in question from the remote server where that value existed(C# method code at end of this message). When I then did a console writeline to show me the value it said it was 6.52. At this point I was happy because obviously it was pulling the right value and it must be getting changed on our local server as it is entered.

enter image description here

At the same time I had changed the number of decimal places for that column in our local server from 2 to 4. When I retrieved that record it showed the value as 6.5199. So at least I am getting the right value. My question is though why would the console writeline be displaying a rounded number? My assumption would be that it would show the value as it is stored not round it?

enter image description here

Does anyone know what is going on here?

C# Method:

public void getSupplierMasterPriceTEST()
        {
            DataTable priceTable = new DataTable();
            string queryString = "{call sproc164407_2053096_666689 ()}";
            OdbcDataAdapter odbcAdapter = new OdbcDataAdapter();
            OdbcCommand command = new OdbcCommand(queryString, odbcConnection);
            try
            {
                odbcAdapter.SelectCommand = command;
                odbcConnection.Open();
                odbcAdapter.Fill(priceTable);
                Console.WriteLine("Adding to SQL Server");
                //serverConnection.copyTableToServer("Supplier_Master_Price", priceTable);
                foreach(DataRow row in priceTable.Rows)
                {
                    foreach (object item in row.ItemArray)
                    {
                        Console.WriteLine(item);
                    }
                }
                //serverConnection.removeDuplicatesSupplerMasterPrice();
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }
            finally
            {
                odbcConnection.Close();
            }
        }
Danrex
  • 1,657
  • 4
  • 31
  • 44
  • 1
    The rounding is certainly happening. What you don't want are the trailing zeroes. That is: one thing is dealing with the value at the numeric level (rounding 6.5199 to 6.52) and another thing at the string level (how the value is being displayed: 6.52 or 6.5200). It seems that you should look at the column of the database (but all the rows would be affected). On the other hand, what is the problem of having 6.5200? Whenever you will display this value (through an application, I guess), you might show the string format as you wish (with/without trailing zeroes). – varocarbas Nov 24 '15 at 21:00
  • Again this is a decision made by the remote server, not by myself. It is called something different, but the actual name I have to use in the code is what I have used. – Danrex Nov 24 '15 at 21:08
  • @varocarbas Can you post this as an answer so that I can mark it as correct please? – Danrex Nov 24 '15 at 21:55
  • Thanks for the offer, but I think that it is enough like this. My comment is clear and the other comments also contribute to clarify some related issues. If you consider that writing an answer might still be better, you can go ahead and do it yourself (I wouldn't mind at all). – varocarbas Nov 24 '15 at 22:08

3 Answers3

3

Looks like you are storing as a double and not a decimal. Doubles use floating point math which varries in precicion based on the size of the number. Decimals are a number with a defined number of decimal places specified.

Never ever ever use a double for money calculations.

Joshua Morgan
  • 678
  • 6
  • 18
  • 2
    ?! This answer doesn't seem to make any sense and has got +2?! Firstly double/decimal behave identically in this situation (up to the 4th decimal position). Secondly there is not rounding problems, just a string-displaying one; nothing to do with the numeric type. – varocarbas Nov 24 '15 at 21:03
  • The value is stored on the remote server as a float, and I am storing it as a decimal. – Danrex Nov 24 '15 at 21:04
  • Locke125 unfortunately it is not up to me how the data is stored on their end so I just have to roll with the punches. – Danrex Nov 24 '15 at 21:05
  • @Danrex As explained in my comment above and in the one to this question: this answer doesn't make any sense and you don't even have a problem (a rounding problem, certainly not). – varocarbas Nov 24 '15 at 21:06
  • I was just reading your comment and agree with you. The problem was with how C# was displaying it. Certainly I am getting the correct value stored in our local database so their isn't a problem in that sense. – Danrex Nov 24 '15 at 21:07
  • Locke125, don't get offended, but how many friends do you have here? Because a completely wrong answer getting +3 is kind of weird. – varocarbas Nov 24 '15 at 21:09
  • 2
    Floating point math is inherently imprecise. You understand it as a bunch of digits strung together and think a number is a number is a number, but that isn't how the computer stores it. The link mentioned by HuntK24 explains more details if you are interested. – Joshua Morgan Nov 24 '15 at 21:11
  • @Locke125 Please, don't try to convert a complete nonsense into a proof of deep knowledge! Who is the one here not understanding what the floating point is? You think that has any effect in the 4th decimal position? Has none!! And additionally and as explained, this is not a rounding problem. The rounding is being done perfectly; this is a string-showing-a-number problem. – varocarbas Nov 24 '15 at 21:13
  • @varocarbas you are correct. But I still found what Locke posted as helpful, even if not directly answering my question. I don't think there is anything untoward happening here other than someone trying to help out. – Danrex Nov 24 '15 at 21:54
  • @Danrex The whole point of all our comments (and the answers and your question) is being helpful to others (and to you). Including wrong information is not helpful; if this wrong information is upvoted, the message would be still worse. If Locke125 realises that he is wrong (because of me or because of any other thing), he should correct his answer; or someone should highlight it (for future readers). My only intention was helping him realise that this was the case. He might even share this information but not showing it as the right answer to your question, because it is not. – varocarbas Nov 24 '15 at 22:03
0

Locke beat me to it. The answer in this question explains it more in depth: Link

More information about the rounding error: Link

One more inspiring article with a .NET explanation: Binary floating point and .NET

Community
  • 1
  • 1
HuntK24
  • 158
  • 2
  • 13
0

Try to change column type in db table to decimal(18,2)

Ori Samara
  • 71
  • 6