We Keep Coding

Comparing two text large files with URLs in Java with external memory only?

I have the following scenario:
url text file A
url text file B
Each file's size is around 4Gb.
I need to calculate:
all urls in A that are not in B
all urls in B that are not in A
All of the Java-diff examples I'm finding online load the entire list in memory (either with a Map or using an MMap solution). My system doesn't have swap and lacks the memory to be able to do this without External-Memory.
Does anyone know of a solution for this?
This project can do huge file sorts without eating up tons of memory https://github.com/lemire/externalsortinginjava
I am looking for something similar, but for generating diffs. I'm going to start by trying to implement this using that project as a baseline.

If system has enough storage, you can do this via DB. For example :
Create an H2 or sqlite DB (data stored on disk, allocate as much
cache as system can afford)
Load text file in tables A and B (create index on 'url' column)
select url from A where URL not in (select distinct url from B)
select url from B where URL not in (select distinct url from A)

Here is a gist of the solution I came up with: https://gist.github.com/nddipiazza/16cb2a0d23ee60a07121893c26065de4
import com.google.common.collect.Sets;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.LineIterator;
import java.io.File;
import java.io.IOException;
import java.util.HashSet;
import java.util.Set;
public class DiffTextFilesUtil {
static public int CHUNK_SIZE = 100000;
static public class DiffResult {
public Set<String> addedVals = new HashSet<>();
public Set<String> removedVals = new HashSet<>();
}
/**
* Gets diff result of two sorted files with each other.
* #param lhs left hand file - sort this using com.google.code.externalsortinginjava:externalsortinginjava:0.2.5
* #param rhs right hand file - sort this using com.google.code.externalsortinginjava:externalsortinginjava:0.2.5
* #return DiffResult.addedVals were added from lhs to rhs. DiffResult.removedVals were removed from lhs to rhs.
* #throws IOException
*/
public static DiffResult diff(File lhs, File rhs) throws IOException {
DiffResult diffResult = new DiffResult();
LineIterator lhsIter = FileUtils.lineIterator(lhs);
LineIterator rhsIter = FileUtils.lineIterator(rhs);
String lhsTop = null;
String rhsTop = null;
while (lhsIter.hasNext()) {
int ct = CHUNK_SIZE;
Set<String> setLhs = Sets.newHashSet();
Set<String> setRhs = Sets.newHashSet();
while (lhsIter.hasNext() && --ct > 0) {
lhsTop = lhsIter.nextLine();
setLhs.add(lhsTop);
}
while (rhsIter.hasNext()) {
if (rhsTop != null && rhsTop.compareTo(lhsTop) > 0) {
break;
} else if (rhsTop != null && rhsTop.compareTo(lhsTop) == 0) {
setRhs.add(rhsTop);
rhsTop = null;
break;
} else if (rhsTop != null) {
setRhs.add(rhsTop);
}
rhsTop = rhsIter.next();
}
if (rhsTop != null) {
setRhs.add(rhsTop);
}
Sets.difference(setLhs, setRhs).copyInto(diffResult.removedVals);
Sets.difference(setRhs, setLhs).copyInto(diffResult.addedVals);
}
return diffResult;
}
}

need advice regarding design and performance issue

I have data retrieved from a server at very high rate, the datas sent are in form of a message that resembles the following format:
$FMSn,par1,par2...,...,...,...,..,...,....,par20 //where n is number ranges from 1 to 12
this message I need to process to parse some data.
but less frequently the server sends other message in different format, that message is not important and could be discarded and the difference between it and the previously described messages in format is that
the previous message starts with $FMS while the other message not.
to distinguish between these messages to know which one is that should be processed, i created a class FMSParser as shown below and i checked if the message header is
$FMS
or not.
my question is, should i create a new object of FMSParser class in the loop in which the messages from the server are received or create one object in the whole
program and in the loop in which the data are recived i just call isValid method and getParam(). in other words in code:
should i choose solution 1 or 2?
solution 1:
loop for messages receiving:
msg = receive message();
fmsParser = new FMSParser(msg);
if (fmsParser.isValid) {
params = fmsParser.getParam();
}
solution 2:
fmsParser = new FMSParser();
loop for messages receiving:
msg = receive message();
if (fmsParser.isValid(msg)) {
params = fmsParser.getParam();
}
code:
private class FMSParser {
private final static String HEADER = "$FMS"
private String[] mSplittedMsg;
FMSParser() {}
public boolean isValidMsg(String msg) {
boolean isValid = false;
this.mSplittedMsg = msg.split(",");
for (int i = 0; i < 12; i++) {
if (splittedMsg[0].equals(HEADER+i)) {
valid = true;
break;
}
}
return valid;
}
public String [] getParam() {
return this.mSplittedMsg;
}
}

If you construct a new FMSParser each time through the loop, it will require memory allocation and garbage collection.
I would choose option 3 which makes the FMSParser immutable, meaning it is thread-safe.
FMSParser fmsParser = new FMSParser();
while (messageIterator.hasNext()) {
String msg = messageIterator.next();
if (fmsParser.isValid(msg)) {
params = fmsParser.getParam(msg);
}
}
Eg:
public class FMSParser {
public boolean isValid(String msg) {
return msg.startsWith("$FMS");
}
public String[] getParams(String msg) {
return msg.split(",");
}
}

Error with ChatColor.translateAlternateColorCodes

I'm getting a error with ChatColor.translateAlternateColorCodes since I added a custom config to my plugin.
Here is the error:
Caused by: java.lang.NullPointerException
at org.bukkit.ChatColor.translateAlternateColorCodes(ChatColor.java:206)
~[spigot.jar:git-Spigot-1473]
at com.gmail.santiagoelheroe.LoginVip.<init>(LoginVip.java:44) ~[?:?]
The error says the problem is at line 44 inside LoginVip class.
YamlConfiguration configuracion = YamlConfiguration.loadConfiguration(configFile);
String textpermisos = configuracion.getString("Configuration.NoPermissionsMessage");
// Line 44
String permisos = (ChatColor.translateAlternateColorCodes('&', textpermisos));
String prefixtext = configuracion.getString("Configuration.Prefix");
String prefix = (ChatColor.translateAlternateColorCodes('&', prefixtext));
I have to fix this error to finish my first plugin.
Config.class:
import java.io.File;
import java.util.logging.Level;
import org.bukkit.configuration.file.YamlConfiguration;
public class Config {
public static File configFile = new File("Plugins/LoginVip/config.yml");
public static void load() {
YamlConfiguration spawn = YamlConfiguration.loadConfiguration(configFile);
}
public static void saveConfig() {
YamlConfiguration configuracion = new YamlConfiguration();
configuracion.set("Configuration.NoPermissionsMessage", "&cYou don't have permissions to do that");
try {
configuracion.save(configFile);
} catch (Exception e) {
LoginVip.log.log(Level.WARNING, "[LV] Error creating Config.yml file");
}
}
}
onEnable:
#Override
public void onEnable() {
log.log(Level.INFO, "[LV] Plugin loaded");
if(!Config.configFile.exists()) {
Config.saveConfig();
}
if(!Config.spawnFile.exists()) {
Config.saveSpawn();
}
Config.load();
}

textpermisos is null. From the Javadocs of MemoryConfiguration.getString(String):
Gets the requested String by path.
If the String does not exist but a default value has been specified, this will return the default value. If the String does not exist and no default value was specified, this will return null.
This means that your configuration file does not contain the key-value mapping for "Configuration.NoPermissionsMessage". It is null, which is then passed into ChatColor.translateAlternateColorCodes(char, String). Here is its source code, with a comment of mine indicating which line ChatColor.java:206 in your crash log was:
/*
* Translates a string using an alternate color code character into a
* string that uses the internal ChatColor.COLOR_CODE color code
* character. The alternate color code character will only be replaced if
* it is immediately followed by 0-9, A-F, a-f, K-O, k-o, R or r.
*
* #param altColorChar The alternate color code character to replace. Ex: &
* #param textToTranslate Text containing the alternate color code character.
* #return Text containing the ChatColor.COLOR_CODE color code character.
*/
public static String translateAlternateColorCodes(char altColorChar, String textToTranslate) {
char[] b = textToTranslate.toCharArray(); // textToTranslate is null, it causes a NPE to be thrown.
for (int i = 0; i < b.length - 1; i++) {
if (b[i] == altColorChar && "0123456789AaBbCcDdEeFfKkLlMmNnOoRr".indexOf(b[i+1]) > -1) {
b[i] = ChatColor.COLOR_CHAR;
b[i+1] = Character.toLowerCase(b[i+1]);
}
}
return new String(b);
}
To solve this:
Add default mapping so getString() would not return null but instead a default value. Here is one way to do this (consult the documentation for applying as HashMap):
YamlConfiguration configuracion = YamlConfiguration.loadConfiguration(configFile);
String defpermisos = "";
String textpermisos = configuracion.getString("Configuration.NoPermissionsMessage", defpermisos);
String permisos = ChatColor.translateAlternateColorCodes('&', textpermisos);
String defprefix = "";
String textprefix = configuracion.getString("Configuration.Prefix", defprefix);
String prefix = ChatColor.translateAlternateColorCodes('&', textprefix);
Modify your code to only translate color codes after a != null check.
YamlConfiguration configuracion = YamlConfiguration.loadConfiguration(configFile);
String textpermisos = configuracion.getString("Configuration.NoPermissionsMessage");
String permisos = null;
if (textpermisos != null)
permisos = ChatColor.translateAlternateColorCodes('&', textpermisos);
String prefixtext = configuracion.getString("Configuration.Prefix");
String prefix = null;
if (prefixtext != null)
prefix = ChatColor.translateAlternateColorCodes('&', prefixtext);

How to log repeated warnings only once

There is a pattern that happens every now and then. I have a method called many times, and it contains this snippet:
Foo foo = getConfiguredFoo();
if (foo == null) {
logger.warn("Foo not configured");
foo = getDefaultFoo();
}
Then my log file is cluttered with this warning a hundred times. I know I can grep it out, but I wonder if there is a better way to see this warning only once.
Note: the duplication of messages is a correct behavior by default, so this is not about avoiding unintentional duplicate log message. I tagged my question as log4j, but I'm open to other java logging frameworks.

Here is what I can come up with: a class that accumulates warnings which can be dumped at the end. It's in groovy, but you can get the point. The dumping part can be customized to use a logger, of course.
class BadNews {
static Map<String,List<Object>> warnings = [:];
static void warn(String key, Object uniqueStuff) {
def knownWarnings = warnings[key]
if (! knownWarnings) {
knownWarnings = []
warnings[key] = knownWarnings
}
knownWarnings << uniqueStuff
}
static void dumpWarnings(PrintStream out) {
warnings.each{key, stuffs ->
out.println("$key: " + stuffs.size())
stuffs.each{
out.println("\t$it")
}
}
}
}
class SomewhereElse {
def foo(Bar bar) {
if (! bar)
BadNews.warn("Empty bar", this)
}
}

I faced a similar problem sometime ago but could not find any way of dealing with this in Log4J.
I finally did the following:
Foo foo = getConfiguredFoo();
if (foo == null) {
if(!warningLogged)logger.warn("Foo not configured");
warningLogged = true
foo = getDefaultFoo();
}
This solution is OK if you have one or two log statements you don't want to see repeated in your logs but does not scale up with more log statements (you need a boolean for every message logged)

You could write a wrapper around your logging to store the last line logged. Depending on how you implement, you could add some sort of counter to log how many times it got logged or you may choose to subclass Logger instead of having an external wrapper. Could be configurable with a boolean suppressDuplicates if you needed that too.
public class LogWrapper{
Logger logger = Logger.getLogger("your logger here");
String lastLine = new String();
public void warn(String s){
if (lastLine.compareToIgnoreCase(s) == 0)
return;
else {
lastLine = s;
logger.warn(s);
}
}
}

If this is the only thing you want to print one time, then using a saved boolean would be your best bet. If you wanted something you could use throughout your project, I have created something that may be useful. I just created a Java class that uses a log4j logger instance. When I want to log a message, I just do something like this:
LogConsolidated.log(logger, Level.WARN, 5000, "File: " + f + " not found.", e);
Instead of:
logger.warn("File: " + f + " not found.", e);
Which makes it log a maximum of 1 time every 5 seconds, and prints how many times it should have logged (e.g. |x53|). Obviously, you can make it so you don't have as many parameters, or pull the level out by doing log.warn or something, but this works for my use case.
For you (if you only want to print one time, every time) this is overkill, but you can still do it by passing in something like: Long.MAX_LONG in as the 3rd parameter. I like the flexibility to be able to determine frequency for each specific log message (hence the parameter). For example, this would accomplish what you want:
LogConsolidated.log(logger, Level.WARN, Long.MAX_LONG, "File: " + f + " not found.", e);
Here is the LogConsolidated class:
import java.util.HashMap;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
public class LogConsolidated {
private static HashMap<String, TimeAndCount> lastLoggedTime = new HashMap<>();
/**
* Logs given <code>message</code> to given <code>logger</code> as long as:
* <ul>
* <li>A message (from same class and line number) has not already been logged within the past <code>timeBetweenLogs</code>.</li>
* <li>The given <code>level</code> is active for given <code>logger</code>.</li>
* </ul>
* Note: If messages are skipped, they are counted. When <code>timeBetweenLogs</code> has passed, and a repeat message is logged,
* the count will be displayed.
* #param logger Where to log.
* #param level Level to log.
* #param timeBetweenLogs Milliseconds to wait between similar log messages.
* #param message The actual message to log.
* #param t Can be null. Will log stack trace if not null.
*/
public static void log(Logger logger, Level level, long timeBetweenLogs, String message, Throwable t) {
if (logger.isEnabledFor(level)) {
String uniqueIdentifier = getFileAndLine();
TimeAndCount lastTimeAndCount = lastLoggedTime.get(uniqueIdentifier);
if (lastTimeAndCount != null) {
synchronized (lastTimeAndCount) {
long now = System.currentTimeMillis();
if (now - lastTimeAndCount.time < timeBetweenLogs) {
lastTimeAndCount.count++;
return;
} else {
log(logger, level, "|x" + lastTimeAndCount.count + "| " + message, t);
}
}
} else {
log(logger, level, message, t);
}
lastLoggedTime.put(uniqueIdentifier, new TimeAndCount());
}
}
private static String getFileAndLine() {
StackTraceElement[] stackTrace = Thread.currentThread().getStackTrace();
boolean enteredLogConsolidated = false;
for (StackTraceElement ste : stackTrace) {
if (ste.getClassName().equals(LogConsolidated.class.getName())) {
enteredLogConsolidated = true;
} else if (enteredLogConsolidated) {
// We have now file/line before entering LogConsolidated.
return ste.getFileName() + ":" + ste.getLineNumber();
}
}
return "?";
}
private static void log(Logger logger, Level level, String message, Throwable t) {
if (t == null) {
logger.log(level, message);
} else {
logger.log(level, message, t);
}
}
private static class TimeAndCount {
long time;
int count;
TimeAndCount() {
this.time = System.currentTimeMillis();
this.count = 0;
}
}
}

How to convert Microsoft Locale ID (LCID) into language code or Locale object in Java

I need to translate a Microsoft locale ID, such as 1033 (for US English), into either an ISO 639 language code or directly into a Java Locale instance. (Edit: or even simply into the "Language - Country/Region" in Microsoft's table.)
Is this possible, and what's the easiest way? Preferably using only JDK standard libraries, of course, but if that's not possible, with a 3rd party library.

You could use GetLocaleInfo to do this (assuming you were running on Windows (win2k+)).
This C++ code demonstrates how to use the function:
#include "windows.h"
int main()
{
HANDLE stdout = GetStdHandle(STD_OUTPUT_HANDLE);
if(INVALID_HANDLE_VALUE == stdout) return 1;
LCID Locale = 0x0c01; //Arabic - Egypt
int nchars = GetLocaleInfoW(Locale, LOCALE_SISO639LANGNAME, NULL, 0);
wchar_t* LanguageCode = new wchar_t[nchars];
GetLocaleInfoW(Locale, LOCALE_SISO639LANGNAME, LanguageCode, nchars);
WriteConsoleW(stdout, LanguageCode, nchars, NULL, NULL);
delete[] LanguageCode;
return 0;
}
It would not take much work to turn this into a JNA call. (Tip: emit constants as ints to find their values.)
Sample JNA code:
draw a Windows cursor
print Unicode on a Windows console
Using JNI is a bit more involved, but is manageable for a relatively trivial task.
At the very least, I would look into using native calls to build your conversion database. I'm not sure if Windows has a way to enumerate the LCIDs, but there's bound to be something in .Net. As a build-level thing, this isn't a huge burden. I would want to avoid manual maintenance of the list.

As it started to look like there is no ready Java solution to do this mapping, we took the ~20 minutes to roll something of our own, at least for now.
We took the information from the horse's mouth, i.e. http://msdn.microsoft.com/en-us/goglobal/bb964664.aspx, and copy-pasted it (through Excel) into a .properties file like this:
1078 = Afrikaans - South Africa
1052 = Albanian - Albania
1118 = Amharic - Ethiopia
1025 = Arabic - Saudi Arabia
5121 = Arabic - Algeria
...
(You can download the file here if you have similar needs.)
Then there's a very simple class that reads the information from the .properties file into a map, and has a method for doing the conversion.
Map<String, String> lcidToDescription;
public String getDescription(String lcid) { ... }
And yes, this doesn't actually map to language code or Locale object (which is what I originally asked), but to Microsoft's "Language - Country/Region" description. It turned out this was sufficient for our current need.
Disclaimer: this really is a minimalistic, "dummy" way of doing it yourself in Java, and obviously keeping (and maintaining) a copy of the LCID mapping information in your own codebase is not very elegant. (On the other hand, neither would I want to include a huge library jar or do anything overly complicated just for this simple mapping.) So despite this answer, feel free to post more elegant solutions or existing libraries if you know of anything like that.

The following code will programmatically create a mapping between Microsoft LCID codes and Java Locales, making it easier to keep the mapping up-to-date:
import java.io.IOException;
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
/**
* #author Gili Tzabari
*/
public final class Locales
{
/**
* Maps a Microsoft LCID to a Java Locale.
*/
private final Map<Integer, Locale> lcidToLocale = new HashMap<>(LcidToLocaleMapping.NUM_LOCALES);
public Locales()
{
// Try loading the mapping from cache
File file = new File("lcid-to-locale.properties");
Properties properties = new Properties();
try (FileInputStream in = new FileInputStream(file))
{
properties.load(in);
for (Object key: properties.keySet())
{
String keyString = key.toString();
Integer lcid = Integer.parseInt(keyString);
String languageTag = properties.getProperty(keyString);
lcidToLocale.put(lcid, Locale.forLanguageTag(languageTag));
}
return;
}
catch (IOException unused)
{
// Cache does not exist or is invalid, regenerate...
lcidToLocale.clear();
}
LcidToLocaleMapping mapping;
try
{
mapping = new LcidToLocaleMapping();
}
catch (IOException e)
{
// Unrecoverable runtime failure
throw new AssertionError(e);
}
for (Locale locale: Locale.getAvailableLocales())
{
if (locale == Locale.ROOT)
{
// Special case that doesn't map to a real locale
continue;
}
String language = locale.getDisplayLanguage(Locale.ENGLISH);
String country = locale.getDisplayCountry(Locale.ENGLISH);
country = mapping.getCountryAlias(country);
String script = locale.getDisplayScript();
for (Integer lcid: mapping.listLcidFor(language, country, script))
{
lcidToLocale.put(lcid, locale);
properties.put(lcid.toString(), locale.toLanguageTag());
}
}
// Cache the mapping
try (FileOutputStream out = new FileOutputStream(file))
{
properties.store(out, "LCID to Locale mapping");
}
catch (IOException e)
{
// Unrecoverable runtime failure
throw new AssertionError(e);
}
}
/**
* #param lcid a Microsoft LCID code
* #return a Java locale
* #see https://msdn.microsoft.com/en-us/library/cc223140.aspx
*/
public Locale fromLcid(int lcid)
{
return lcidToLocale.get(lcid);
}
}
import com.google.common.collect.HashMultimap;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.SetMultimap;
import com.google.common.collect.Sets;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import org.bitbucket.cowwoc.preconditions.Preconditions;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Generates a mapping between Microsoft LCIDs and Java Locales.
* <p>
* #see http://stackoverflow.com/a/32324060/14731
* #author Gili Tzabari
*/
final class LcidToLocaleMapping
{
private static final int NUM_COUNTRIES = 194;
private static final int NUM_LANGUAGES = 13;
private static final int NUM_SCRIPTS = 5;
/**
* The number of locales we are expecting. This value is only used for performance optimization.
*/
public static final int NUM_LOCALES = 238;
private static final List<String> EXPECTED_HEADERS = ImmutableList.of("lcid", "language", "location");
// [language] - [comment] ([script])
private static final Pattern languagePattern = Pattern.compile("^(.+?)(?: - (.*?))?(?: \\((.+)\\))?$");
/**
* Maps a country to a list of entries.
*/
private static final SetMultimap<String, Mapping> COUNTRY_TO_ENTRIES = HashMultimap.create(NUM_COUNTRIES,
NUM_LOCALES / NUM_COUNTRIES);
/**
* Maps a language to a list of entries.
*/
private static final SetMultimap<String, Mapping> LANGUAGE_TO_ENTRIES = HashMultimap.create(NUM_LANGUAGES,
NUM_LOCALES / NUM_LANGUAGES);
/**
* Maps a language script to a list of entries.
*/
private static final SetMultimap<String, Mapping> SCRIPT_TO_ENTRIES = HashMultimap.create(NUM_SCRIPTS,
NUM_LOCALES / NUM_SCRIPTS);
/**
* Maps a Locale country name to a LCID country name.
*/
private static final Map<String, String> countryAlias = ImmutableMap.<String, String>builder().
put("United Arab Emirates", "U.A.E.").
build();
/**
* A mapping between a country, language, script and LCID.
*/
private static final class Mapping
{
public final String country;
public final String language;
public final String script;
public final int lcid;
Mapping(String country, String language, String script, int lcid)
{
Preconditions.requireThat(country, "country").isNotNull();
Preconditions.requireThat(language, "language").isNotNull().isNotEmpty();
Preconditions.requireThat(script, "script").isNotNull();
this.country = country;
this.language = language;
this.script = script;
this.lcid = lcid;
}
#Override
public int hashCode()
{
return country.hashCode() + language.hashCode() + script.hashCode() + lcid;
}
#Override
public boolean equals(Object obj)
{
if (!(obj instanceof Locales))
return false;
Mapping other = (Mapping) obj;
return country.equals(other.country) && language.equals(other.language) && script.equals(other.script) &&
lcid == other.lcid;
}
}
private final Logger log = LoggerFactory.getLogger(LcidToLocaleMapping.class);
/**
* Creates a new LCID to Locale mapping.
* <p>
* #throws IOException if an I/O error occurs while reading the LCID table
*/
LcidToLocaleMapping() throws IOException
{
Document doc = Jsoup.connect("https://msdn.microsoft.com/en-us/library/cc223140.aspx").get();
Element mainBody = doc.getElementById("mainBody");
Elements elements = mainBody.select("table");
assert (elements.size() == 1): elements;
for (Element table: elements)
{
boolean firstRow = true;
for (Element row: table.select("tr"))
{
if (firstRow)
{
// Make sure that columns are ordered as expected
List<String> headers = new ArrayList<>(3);
Elements columns = row.select("th");
for (Element column: columns)
headers.add(column.text().toLowerCase());
assert (headers.equals(EXPECTED_HEADERS)): headers;
firstRow = false;
continue;
}
Elements columns = row.select("td");
assert (columns.size() == 3): columns;
Integer lcid = Integer.parseInt(columns.get(0).text(), 16);
Matcher languageMatcher = languagePattern.matcher(columns.get(1).text());
if (!languageMatcher.find())
throw new AssertionError();
String language = languageMatcher.group(1);
String script = languageMatcher.group(2);
if (script == null)
script = "";
String country = columns.get(2).text();
Mapping mapping = new Mapping(country, language, script, lcid);
COUNTRY_TO_ENTRIES.put(country, mapping);
LANGUAGE_TO_ENTRIES.put(language, mapping);
if (!script.isEmpty())
SCRIPT_TO_ENTRIES.put(script, mapping);
}
}
}
/**
* Returns the LCID codes associated with a [country, language, script] combination.
* <p>
* #param language a language
* #param country a country (empty string if any country should match)
* #param script a language script (empty string if any script should match)
* #return an empty list if no matches are found
* #throws NullPointerException if any of the arguments are null
* #throws IllegalArgumentException if language is empty
*/
public Collection<Integer> listLcidFor(String language, String country, String script)
throws NullPointerException, IllegalArgumentException
{
Preconditions.requireThat(language, "language").isNotNull().isNotEmpty();
Preconditions.requireThat(country, "country").isNotNull();
Preconditions.requireThat(script, "script").isNotNull();
Set<Mapping> result = LANGUAGE_TO_ENTRIES.get(language);
if (result == null)
{
log.warn("Language '" + language + "' had no corresponding LCID");
return Collections.emptyList();
}
if (!country.isEmpty())
{
Set<Mapping> entries = COUNTRY_TO_ENTRIES.get(country);
result = Sets.intersection(result, entries);
}
if (!script.isEmpty())
{
Set<Mapping> entries = SCRIPT_TO_ENTRIES.get(script);
result = Sets.intersection(result, entries);
}
return result.stream().map(entry -> entry.lcid).collect(Collectors.toList());
}
/**
* #param name the locale country name
* #return the LCID country name
*/
public String getCountryAlias(String name)
{
String result = countryAlias.get(name);
if (result == null)
return name;
return result;
}
}
Maven dependencies:
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>18.0</version>
</dependency>
<dependency>
<groupId>org.bitbucket.cowwoc</groupId>
<artifactId>preconditions</artifactId>
<version>1.25</version>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.8.3</version>
</dependency>
Usage:
System.out.println("Language: " + new Locales().fromLcid(1033).getDisplayLanguage());
will print "Language: English".
Meaning, LCID 1033 maps to the English language.
NOTE: This only generates mappings for locales available on your runtime JVM. Meaning, you will only get a subset of all possible Locales. That said, I don't think it is technically possible to instantiate Locales that your JVM doesn't support, so this is probably the best we can do...

The was the first hit on google for "Java LCID" is this javadoc:
gnu.java.awt.font.opentype.NameDecoder
private static java.util.Locale
getWindowsLocale(int lcid)
Maps a Windows LCID into a Java Locale.
Parameters:
lcid - the Windows language ID whose Java locale is to be retrieved.
Returns:
an suitable Locale, or null if the mapping cannot be performed.
I'm not sure where to go about downloading this library, but it's GNU, so it shouldn't be too hard to find.

Here is a script to paste into the F12 console and extract the mapping for the currently 273 languages to their lcid (to be used on https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/a9eac961-e77d-41a6-90a5-ce1a8b0cdb9c):
// extract data from https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/a9eac961-e77d-41a6-90a5-ce1a8b0cdb9c
const locales = {}, dataTable = document.querySelector('div.table-scroll-wrapper:nth-of-type(2)>table.protocol-table');
for (let i=1, l=dataTable.rows.length; i<l; i++) {
const row = dataTable.rows[i];
let locale = Number(row.cells[2].textContent.trim()); // hex to decimal
let name = row.cells[3].textContent.trim(); // cc-LL
if ((locale > 1024) && (name.indexOf('-') > 0)) // only cc-LL (languages, not countries)
locales[locale] = name;
}
console.table(locales); // 273 entries