Most libraries give you arbitrarily large precision, GMP included. However even with large precision there are some numbers that cannot be represented exactly in binary format, much same as you cannot represent 1/3 in decimal. For many applications setting the precision to a high number, like 10, doing the calculations, and then rounding the results back to desired precision, like 3 works. Would it not work for you? See this - Is there a C++ equivalent to Java's BigDecimal?
You could also use http://software.intel.com/en-us/articles/intel-decimal-floating-point-math-library
********* EDITS
Exact representation DOES NOT exist in binary floating point for many numbers; the kind that most current floating point libraries offer. A number like 0.1 CANNOT be represented as a binary number, whatever be the precision.
To be able to do what you suggest the library would have to do equivalent of 'hand addition', 'hand division' - the kind you do on pencil and paper to add two decimal numbers. e.g. to store 0.1, the library might elect to represent it as a string itself and then do additions on strings. Needless to say a naive implementation would make the process exceedingly slow - orders of magnitude slow. To add 0.1 + 0.1, it would have to parse string, add 1+1, remember the carries, remember the decimal position etc. That is the thing that the computer micro code does for you in few CPU cycles (or a single instruction). Instead of single instruction, your software library would end up taking like 100 CPU cycles/instructions.
If it tries to convert 0.1 to a number, it is back to square 1 - 0.1 cannot be a number in binary.
However people do recognize the need for representing 0.1 exactly. It is just that binary number representation is NOT going to do it. That is where newer floating point standards come in, and that is where the intel decimal point library is headed.
Repeating my previous example, suppose you had a 10 base computer that could do 10 base numbers. that computer cannot store 1/3 as a 'plain' floating point number. It would have to store the representation that the number is 1/3. The equivalent of how it is written on paper. Try writing 1/3 as a base 10 floating point number on paper.
Also see Why can't decimal numbers be represented exactly in binary?