I need to translate a Microsoft locale ID, such as 1033 (for US English), into either an ISO 639 language code or directly into a Java Locale instance. (Edit: or even simply into the "Language - Country/Region" in Microsoft's table.)
Is this possible, and what's the easiest way? Preferably using only JDK standard libraries, of course, but if that's not possible, with a 3rd party library.
You could use GetLocaleInfo to do this (assuming you were running on Windows (win2k+)).
This C++ code demonstrates how to use the function:
#include "windows.h"
int main()
{
HANDLE stdout = GetStdHandle(STD_OUTPUT_HANDLE);
if(INVALID_HANDLE_VALUE == stdout) return 1;
LCID Locale = 0x0c01; //Arabic - Egypt
int nchars = GetLocaleInfoW(Locale, LOCALE_SISO639LANGNAME, NULL, 0);
wchar_t* LanguageCode = new wchar_t[nchars];
GetLocaleInfoW(Locale, LOCALE_SISO639LANGNAME, LanguageCode, nchars);
WriteConsoleW(stdout, LanguageCode, nchars, NULL, NULL);
delete[] LanguageCode;
return 0;
}
It would not take much work to turn this into a JNA call. (Tip: emit constants as ints to find their values.)
Sample JNA code:
Using JNI is a bit more involved, but is manageable for a relatively trivial task.
At the very least, I would look into using native calls to build your conversion database. I'm not sure if Windows has a way to enumerate the LCIDs, but there's bound to be something in .Net. As a build-level thing, this isn't a huge burden. I would want to avoid manual maintenance of the list.
As it started to look like there is no ready Java solution to do this mapping, we took the ~20 minutes to roll something of our own, at least for now.
We took the information from the horse's mouth, i.e. http://msdn.microsoft.com/en-us/goglobal/bb964664.aspx, and copy-pasted it (through Excel) into a .properties file like this:
1078 = Afrikaans - South Africa
1052 = Albanian - Albania
1118 = Amharic - Ethiopia
1025 = Arabic - Saudi Arabia
5121 = Arabic - Algeria
...
(You can download the file here if you have similar needs.)
Then there's a very simple class that reads the information from the .properties file into a map, and has a method for doing the conversion.
Map<String, String> lcidToDescription;
public String getDescription(String lcid) { ... }
And yes, this doesn't actually map to language code or Locale object (which is what I originally asked), but to Microsoft's "Language - Country/Region" description. It turned out this was sufficient for our current need.
Disclaimer: this really is a minimalistic, "dummy" way of doing it yourself in Java, and obviously keeping (and maintaining) a copy of the LCID mapping information in your own codebase is not very elegant. (On the other hand, neither would I want to include a huge library jar or do anything overly complicated just for this simple mapping.) So despite this answer, feel free to post more elegant solutions or existing libraries if you know of anything like that.
The following code will programmatically create a mapping between Microsoft LCID codes and Java Locales, making it easier to keep the mapping up-to-date:
import java.io.IOException;
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
/**
* @author Gili Tzabari
*/
public final class Locales
{
/**
* Maps a Microsoft LCID to a Java Locale.
*/
private final Map<Integer, Locale> lcidToLocale = new HashMap<>(LcidToLocaleMapping.NUM_LOCALES);
public Locales()
{
// Try loading the mapping from cache
File file = new File("lcid-to-locale.properties");
Properties properties = new Properties();
try (FileInputStream in = new FileInputStream(file))
{
properties.load(in);
for (Object key: properties.keySet())
{
String keyString = key.toString();
Integer lcid = Integer.parseInt(keyString);
String languageTag = properties.getProperty(keyString);
lcidToLocale.put(lcid, Locale.forLanguageTag(languageTag));
}
return;
}
catch (IOException unused)
{
// Cache does not exist or is invalid, regenerate...
lcidToLocale.clear();
}
LcidToLocaleMapping mapping;
try
{
mapping = new LcidToLocaleMapping();
}
catch (IOException e)
{
// Unrecoverable runtime failure
throw new AssertionError(e);
}
for (Locale locale: Locale.getAvailableLocales())
{
if (locale == Locale.ROOT)
{
// Special case that doesn't map to a real locale
continue;
}
String language = locale.getDisplayLanguage(Locale.ENGLISH);
String country = locale.getDisplayCountry(Locale.ENGLISH);
country = mapping.getCountryAlias(country);
String script = locale.getDisplayScript();
for (Integer lcid: mapping.listLcidFor(language, country, script))
{
lcidToLocale.put(lcid, locale);
properties.put(lcid.toString(), locale.toLanguageTag());
}
}
// Cache the mapping
try (FileOutputStream out = new FileOutputStream(file))
{
properties.store(out, "LCID to Locale mapping");
}
catch (IOException e)
{
// Unrecoverable runtime failure
throw new AssertionError(e);
}
}
/**
* @param lcid a Microsoft LCID code
* @return a Java locale
* @see https://msdn.microsoft.com/en-us/library/cc223140.aspx
*/
public Locale fromLcid(int lcid)
{
return lcidToLocale.get(lcid);
}
}
import com.google.common.collect.HashMultimap;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.SetMultimap;
import com.google.common.collect.Sets;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import org.bitbucket.cowwoc.preconditions.Preconditions;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Generates a mapping between Microsoft LCIDs and Java Locales.
* <p>
* @see http://stackoverflow.com/a/32324060/14731
* @author Gili Tzabari
*/
final class LcidToLocaleMapping
{
private static final int NUM_COUNTRIES = 194;
private static final int NUM_LANGUAGES = 13;
private static final int NUM_SCRIPTS = 5;
/**
* The number of locales we are expecting. This value is only used for performance optimization.
*/
public static final int NUM_LOCALES = 238;
private static final List<String> EXPECTED_HEADERS = ImmutableList.of("lcid", "language", "location");
// [language] - [comment] ([script])
private static final Pattern languagePattern = Pattern.compile("^(.+?)(?: - (.*?))?(?: \\((.+)\\))?$");
/**
* Maps a country to a list of entries.
*/
private static final SetMultimap<String, Mapping> COUNTRY_TO_ENTRIES = HashMultimap.create(NUM_COUNTRIES,
NUM_LOCALES / NUM_COUNTRIES);
/**
* Maps a language to a list of entries.
*/
private static final SetMultimap<String, Mapping> LANGUAGE_TO_ENTRIES = HashMultimap.create(NUM_LANGUAGES,
NUM_LOCALES / NUM_LANGUAGES);
/**
* Maps a language script to a list of entries.
*/
private static final SetMultimap<String, Mapping> SCRIPT_TO_ENTRIES = HashMultimap.create(NUM_SCRIPTS,
NUM_LOCALES / NUM_SCRIPTS);
/**
* Maps a Locale country name to a LCID country name.
*/
private static final Map<String, String> countryAlias = ImmutableMap.<String, String>builder().
put("United Arab Emirates", "U.A.E.").
build();
/**
* A mapping between a country, language, script and LCID.
*/
private static final class Mapping
{
public final String country;
public final String language;
public final String script;
public final int lcid;
Mapping(String country, String language, String script, int lcid)
{
Preconditions.requireThat(country, "country").isNotNull();
Preconditions.requireThat(language, "language").isNotNull().isNotEmpty();
Preconditions.requireThat(script, "script").isNotNull();
this.country = country;
this.language = language;
this.script = script;
this.lcid = lcid;
}
@Override
public int hashCode()
{
return country.hashCode() + language.hashCode() + script.hashCode() + lcid;
}
@Override
public boolean equals(Object obj)
{
if (!(obj instanceof Locales))
return false;
Mapping other = (Mapping) obj;
return country.equals(other.country) && language.equals(other.language) && script.equals(other.script) &&
lcid == other.lcid;
}
}
private final Logger log = LoggerFactory.getLogger(LcidToLocaleMapping.class);
/**
* Creates a new LCID to Locale mapping.
* <p>
* @throws IOException if an I/O error occurs while reading the LCID table
*/
LcidToLocaleMapping() throws IOException
{
Document doc = Jsoup.connect("https://msdn.microsoft.com/en-us/library/cc223140.aspx").get();
Element mainBody = doc.getElementById("mainBody");
Elements elements = mainBody.select("table");
assert (elements.size() == 1): elements;
for (Element table: elements)
{
boolean firstRow = true;
for (Element row: table.select("tr"))
{
if (firstRow)
{
// Make sure that columns are ordered as expected
List<String> headers = new ArrayList<>(3);
Elements columns = row.select("th");
for (Element column: columns)
headers.add(column.text().toLowerCase());
assert (headers.equals(EXPECTED_HEADERS)): headers;
firstRow = false;
continue;
}
Elements columns = row.select("td");
assert (columns.size() == 3): columns;
Integer lcid = Integer.parseInt(columns.get(0).text(), 16);
Matcher languageMatcher = languagePattern.matcher(columns.get(1).text());
if (!languageMatcher.find())
throw new AssertionError();
String language = languageMatcher.group(1);
String script = languageMatcher.group(2);
if (script == null)
script = "";
String country = columns.get(2).text();
Mapping mapping = new Mapping(country, language, script, lcid);
COUNTRY_TO_ENTRIES.put(country, mapping);
LANGUAGE_TO_ENTRIES.put(language, mapping);
if (!script.isEmpty())
SCRIPT_TO_ENTRIES.put(script, mapping);
}
}
}
/**
* Returns the LCID codes associated with a [country, language, script] combination.
* <p>
* @param language a language
* @param country a country (empty string if any country should match)
* @param script a language script (empty string if any script should match)
* @return an empty list if no matches are found
* @throws NullPointerException if any of the arguments are null
* @throws IllegalArgumentException if language is empty
*/
public Collection<Integer> listLcidFor(String language, String country, String script)
throws NullPointerException, IllegalArgumentException
{
Preconditions.requireThat(language, "language").isNotNull().isNotEmpty();
Preconditions.requireThat(country, "country").isNotNull();
Preconditions.requireThat(script, "script").isNotNull();
Set<Mapping> result = LANGUAGE_TO_ENTRIES.get(language);
if (result == null)
{
log.warn("Language '" + language + "' had no corresponding LCID");
return Collections.emptyList();
}
if (!country.isEmpty())
{
Set<Mapping> entries = COUNTRY_TO_ENTRIES.get(country);
result = Sets.intersection(result, entries);
}
if (!script.isEmpty())
{
Set<Mapping> entries = SCRIPT_TO_ENTRIES.get(script);
result = Sets.intersection(result, entries);
}
return result.stream().map(entry -> entry.lcid).collect(Collectors.toList());
}
/**
* @param name the locale country name
* @return the LCID country name
*/
public String getCountryAlias(String name)
{
String result = countryAlias.get(name);
if (result == null)
return name;
return result;
}
}
Maven dependencies:
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>18.0</version>
</dependency>
<dependency>
<groupId>org.bitbucket.cowwoc</groupId>
<artifactId>preconditions</artifactId>
<version>1.25</version>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.8.3</version>
</dependency>
Usage:
System.out.println("Language: " + new Locales().fromLcid(1033).getDisplayLanguage());
will print "Language: English".
Meaning, LCID 1033 maps to the English language.
NOTE: This only generates mappings for locales available on your runtime JVM. Meaning, you will only get a subset of all possible Locales. That said, I don't think it is technically possible to instantiate Locales that your JVM doesn't support, so this is probably the best we can do...
The was the first hit on google for "Java LCID" is this javadoc:
gnu.java.awt.font.opentype.NameDecoder
private static java.util.Locale getWindowsLocale(int lcid)
Maps a Windows LCID into a Java Locale. Parameters: lcid - the Windows language ID whose Java locale is to be retrieved. Returns: an suitable Locale, or null if the mapping cannot be performed.
I'm not sure where to go about downloading this library, but it's GNU, so it shouldn't be too hard to find.
Here is a script to paste into the F12 console and extract the mapping for the currently 273 languages to their lcid (to be used on https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/a9eac961-e77d-41a6-90a5-ce1a8b0cdb9c):
// extract data from https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/a9eac961-e77d-41a6-90a5-ce1a8b0cdb9c
const locales = {}, dataTable = document.querySelector('div.table-scroll-wrapper:nth-of-type(2)>table.protocol-table');
for (let i=1, l=dataTable.rows.length; i<l; i++) {
const row = dataTable.rows[i];
let locale = Number(row.cells[2].textContent.trim()); // hex to decimal
let name = row.cells[3].textContent.trim(); // cc-LL
if ((locale > 1024) && (name.indexOf('-') > 0)) // only cc-LL (languages, not countries)
locales[locale] = name;
}
console.table(locales); // 273 entries
It is easily possible with Apache POI using the org.apache.poi.util.LocaleID class.
Java Locale to LCID:
LocaleID.lookupByLanguageTag(new Locale("en", "US").toLanguageTag()).getLcid()
LCID to Java Locale:
new Locale(LocaleID.lookupByLcid(1033).getLanguageTag())
