Flask-Babel Choose right locale using Accept-Language header

Question

Say I have an application with a few translations available:

AVAILABLE_LOCALES = ['en_US', 'en_EN', 'de_AT']

and I'd like to select the most appropriate according to the Accept-Language header.

Using request.accept_languages.best_match

To get the locale, I would write something like this:

@babel.localeselector
def get_locale():
    user = getattr(g, 'user', None)
    if user is not None:
        return user.locale
    return request.accept_languages.best_match(AVAILABLE_LOCALES)

The issue is if a user agent asks for de_DE, the match fails and the choice falls back on the default value (english), while I would rather get de_AT, obviously.

Note: the example in the doc page reads

return request.accept_languages.best_match(['de', 'fr', 'en'])

but this does not allow two different versions of the same language (en_EN, en_US).

Using negotiate_locale

In this GitHub issue, someone suggests the use of Babel's negotiate_locale function:

@babel.localeselector
def get_locale():
    user = getattr(g, 'user', None)
    if user is not None:
        return user.locale
    preferred = [x.replace('-', '_') for x in request.accept_languages.values()]
    return negotiate_locale(preferred, AVAILABLE_LOCALES)

But AFAIU, this does not fix all use cases. Only the case where the user agent asks for ['de_AT', 'de'] when available locales are ['en_US', 'en_EN', 'de_DE']. But not the other way around:

negotiate_locale(['de_AT', 'de'], ['en_US', 'en_EN', 'de_DE'])  # returns de_DE
negotiate_locale(['de_DE', 'de'], ['en_US', 'en_EN', 'de_AT'])  # returns None

The first works because a hardcoded list of aliases makes de point to de_DE.

Also, the case where a browser would only send de_AT and not de is broken as well, but I'm not sure this is a common configuration (see next paragraph).

Is there any reliable way to deal with this?

Typical values in modern browsers?

As a subquestion, I'd like to know what kind of values I can expect from a modern browser.

Is it safe to assume a modern browser, unless wrongly user-customized, will send an Accept-Language of the form [lang+territory, lang] like this?

['en-US', 'en']

Also, I tried my Firefox version to see what I'd get, and it is a bit surprising.

The locale on my system is French:

locale
LANG=fr_FR.UTF-8
LANGUAGE=
LC_CTYPE="fr_FR.UTF-8"
LC_NUMERIC="fr_FR.UTF-8"
LC_TIME="fr_FR.UTF-8"
LC_COLLATE="fr_FR.UTF-8"
LC_MONETARY="fr_FR.UTF-8"
LC_MESSAGES="fr_FR.UTF-8"
LC_PAPER="fr_FR.UTF-8"
LC_NAME="fr_FR.UTF-8"
LC_ADDRESS="fr_FR.UTF-8"
LC_TELEPHONE="fr_FR.UTF-8"
LC_MEASUREMENT="fr_FR.UTF-8"
LC_IDENTIFICATION="fr_FR.UTF-8"
LC_ALL=

The about:config page says:

general.useragent.locale: en-US
intl.accept_languages: fr, fr-fr, en-us, en

and when I connect to my test Flask app, I get

print(request.accept_languages.values())
# ['en_US', 'en']

so the suggested @babel.localeselector would return an English locale.

I get the same values using Chromium.

Are my browsers misconfigured?

Flask-Babel Choose right locale using Accept-Language header

Using request.accept_languages.best_match

Using negotiate_locale

Typical values in modern browsers?

0 Answers0