How to parse numbers like "3.14" with scanf when locale expects "3,14"

Question

Let's say I have to read a file, containing a bunch of floating-point numbers. The numbers can be like 1e+10, 5, -0.15 etc., i.e., any generic floating-point number, using decimal points (this is fixed!). However, my code is a plugin for another application, and I have no control over what's the current locale. It may be Russian, for example, and the LC_NUMERIC rules there call for a decimal comma to be used. Thus, Pi is expected to be spelled as "3,1415...", and

sscanf("3.14", "%f", &x);

returns "1", and x contains "3.0", since it refuses to parse past the '.' in the string.

I need to ignore the locale for such number-parsing tasks.

How does one do that?

I could write a parseFloat function, but this seems like a waste.
I could also save the current locale, reset it temporarily to "C", read the file, and restore to the saved one. What are the performance implications of this? Could setlocale() be very slow on some OS/libc combo, what does it really do under the hood?
Yet another way would be to use iostreams, but again their performance isn't stellar.

let's start with simple stuff, that can't be found by googling: what type is x? — atk, Dec 17 '12 at 18:29
@atk: This is not at all "simple stuff". I think you missed the whole point of the question. — R.. GitHub STOP HELPING ICE, Dec 17 '12 at 18:33
@atk: it doesn't matter for the purposes of the question, but it's a float. — anrieff, Dec 17 '12 at 18:38
@anrieff: of course it matters! If the type won't store the value you want it to, you won't get the result you want! — atk, Dec 17 '12 at 18:54
@R: Yeah, I probably did. But before I spend my time researching something, I'd rather make sure that the basics are covered. — atk, Dec 17 '12 at 18:56
@atk: it's hard to make this kind of an error these days — the compilers detect it and produce a warning :) — anrieff, Dec 17 '12 at 19:03
possible duplicate of [C: What is the portable/safe(thread aware) way to convert a number to a string w/o locale settings?](http://stackoverflow.com/questions/5980100/c-what-is-the-portable-safethread-aware-way-to-convert-a-number-to-a-string-w) — sleske, Jul 25 '13 at 21:05

score 8 · Answer 1 · edited May 03 '22 at 08:38

My personal preference is to never use LC_NUMERIC, i.e. just call setlocale with other categories, or, after calling setlocale with LC_ALL, use setlocale(LC_NUMERIC, "C");. Otherwise, you're completely out of luck if you want to use the standard library for printing or parsing numbers in a standared form for interchange.

If you're lucky enough to be on a POSIX 2008 conforming system, you can use the uselocale and *_l family of functions to make the situation somewhat better. There are at least 2 basic approaches:

Leave the default locale unset (at least the troublesome parts like LC_NUMERIC; LC_CTYPE should probably always be set), and pass a locale_t object for the user's locale to the appropriate *_l functions only when you want to present things to the user in a way that meets their own cultural expectations; otherwise use the default C locale.
Have your code that needs to work with data for interchange keep around a locale_t object for the C locale, and either switch back and forth using uselocale when you need to work with data in a standard form for interchange, or use the appropriate *_l functions (but there is no scanf_l).

Note that implementing your own floating point parser is not easy and is probably not the right solution to the problem unless you're an expert in numerical computing. Getting it right is very hard.

POSIX.1-2008 specifies isalnum_l(), isalpha_l(), isblank_l(), iscntrl_l(), isdigit_l(), isgraph_l(), islower_l(), isprint_l(), ispunct_l(), isspace_l(), isupper_l(), and isxdigit_l().

Yes, but as I said, I don't ever call setlocale() in my code. It's a plugin to a larger program, which in turn has set the locale beforehand. I can't do anything about it. If I don't preserve their locale as my plugin returns control to the original app, other breakages will inevitably occur. — anrieff, Dec 17 '12 at 18:46
Using the locale functions works in threaded systems; using `setlocale()` could mess up other threads working in parallel. — Jonathan Leffler, Dec 17 '12 at 19:01
@anrieff: There's no safe way to save and restore the locale in plain C; when C11 added threads they neglected to add a thread-safe way to do this, leaving a gaping deficiency. If you have the `uselocale` function and related stuff, just use that. — R.. GitHub STOP HELPING ICE, Dec 17 '12 at 19:21

Martin York · Answer 2 · 2013-12-10T19:34:07.547

3

I am not sure how to solve it in C.

But C++ streams (can) have a unique locale object.

std::stringstream  dataStream;
dataStream.imbue(std::locale("C"));

// Note: You must imbue the stream before you do anything wit it.
//       If any operations have been performed then an imbue() can
//       be silently ignored by the stream (which is a pain to debug).

dataStream << "3.14";
float   x;
dataStream >> x;

edited Dec 10 '13 at 19:34

answered Dec 17 '12 at 19:16

Martin York

257,169
86
333
562

Thanks for this post, helped me a lot with another problem! But is there a typo? I guess it's `std::locale("C")`, with lowercase 'L' (`` header). – ollo Dec 10 '13 at 17:55
@ollo: Yes. Corrected. – Martin York Dec 10 '13 at 19:34
1

Prefer `std::locale::classic()` over `std::locale("C")`. – Helmut Grohne Dec 13 '17 at 11:13

score 3 · Accepted Answer · answered Dec 17 '12 at 20:11

Here's what I've done with this stuff in the past.

The goal is to use locale-dependent numeric converters with a C-locale numeric representation. The ideal, of course, would be to use non-locale-dependent converters, or not change the locale, etc., etc., but sometimes you just have to live with what you've got. Locale support is seriously broken in several ways and this is one of them.</rant>

First, extract the number as a string using something like the C grammar's simple pattern for numeric preprocessing tokens. For use with scanf, I do an even simpler one:

" %1[-+0-9.]%[-+0-9A-Za-z.]"

This could be simplified even more, depending on how what else you might expect in the input stream. The only thing you need to do is to not read beyond the end of the number; as long as you don't allow numbers to be followed immediately by letters, without intervening whitespace, the above will work fine.

Now, get the struct lconv (man 7 locale) representing the current locale using localeconv(3). The first entry in that struct is const char* decimal_point; replace all of the '.' characters in your string with that value. (You might also need to replace '+' and '-' characters, although most locales don't change them, and the sign fields in the lconv struct are documented as only applying to currency conversions.) Finally, feed the resulting string through strtod and see if it passes.

This is not a perfect algorithm, particularly since it's not always easy to know how locale-compliant a given library actually is, so you might want to do some autoconf stuff to configure it for the library you're actually compiling with.

using `0-9` etc. in scanf invokes implementation-defined behaviour — M.M, Apr 12 '16 at 08:42

How to parse numbers like "3.14" with scanf when locale expects "3,14"

3 Answers3

Linked