System.Double and System.Float are base 2 floating point types. There are many finite decimal values that have an infinite representation in base 2, much as 1/3 has an infinite representation in base 10. Therefore, when you round to such a value, the binary representation is approximate. To avoid this problem, use the decimal type, which is a base 10 floating point type.
There must be 100 duplicates of this question on stackoverflow, but I am on my phone, which makes it inconvenient to find them and link to them.
For more information, look at the wikipedia article for IEEE double.
Many will say that doubles are "not exact", which is false. Every double value represents an exact value that can be represented exactly in base 10 (except NaN and infinity, of course). That's because 2 is one of the prime factors of 10. The only approximation is when you try to represent certain decimal fractions (or other rational numbers whose denominator has at least one prime factor other than 2).
The best way to understand this, for me at least, is to work out, on paper, the binary representations of a few fractions. For example, try 0.5, 0.625, 3.25, 5/16, 1/3, 0.2, and 0.3.