0

I have two fields in my database that are defined as floats

[Longitude] [float] NULL
[Latitude] [float] NULL

I have added them to my linq2sql dbml and they show as floats as follows enter image description here

I can see in my database there is the value -3.6607431 for Longitude however when I profile the sql (using sql express and a custom profiler tool) I see that the actual value passed in queries is 3.6607429981231689.

It seems to have lost the negative sign and had extra values added.

Is there something I'm missing does a float in SQL not map to a float like this in the dbml or is there another issue going on

Daniel Powell
  • 8,143
  • 11
  • 61
  • 108

2 Answers2

1

A value of -3.6607431f in C# is actually exactly -3.6607429981231689453125 - i.e. that's the exactly-representable float value which is closest to -3.6607431. So I think that answers the precision part - it's just a natural part of binary floating point. Basically, you shouldn't worry about that - unless you need precise decimal values, in which case you should map to decimal instead of float (and update your database schema appropriately).

The sign issue is somewhat different - and my guess is that you've misdiagnosed it somewhere, or you've got some code which negates things. I very much doubt that that's a database problem.

It's not clear from your description whether you're fetching results or passing a parameter in a query (you talk about a "value passed in queries" which could mean either way). If you could show the code involved and how you're diagnosing the problem, that would help.

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • I ended up changing the type to be a decimal in the database and this seemed to get rid of most of the problems, I'm guessing that lat/long shouldnt be stored as float as we actually care exactly what the values are, close enough isnt good enough in this case. – Daniel Powell Mar 14 '12 at 03:38
1

Floating point numbers are only accurate to a certain degree, and have the 'problem' you describe of having 'extra values'. This is how float works.

You should consider using decimal, or possibly double. Further reading: What is the difference between Decimal, Float and Double in C#?

Community
  • 1
  • 1