In my use case, the table in Iceberg format is created. It only receives APPEND operations as it is about recording events in a time series stream. To evaluate the use of the Iceberg format in this use-case, I created a simple Java program that creates a set of 27600 lines. Both the metadata and the parquet file were created but I can't access them via the Java API (https://iceberg.apache.org/docs/latest/java-api-quickstart/). I'm using HadoopCatalog and FileAppender<GenericRecord>. It is important to say that I can read the Parquet file created using pyarrow and datafusion modules via Python 3 script, and it is correct!
I believe that the execution of some method in my program that links the generated Parquet file to the table created in the catalog must be missing.
NOTE: I'm only using Apache Iceberg's Java API in version 1.0.0
There is an org.apache.iceberg.Transaction object in the API that accepts an org.apache.iceberg.DataFile but I haven't seen examples of how to use it and I don't know if it's useful to solve this problem either.
See the program below:
import org.apache.hadoop.conf.Configuration;
import org.apache.iceberg.*;
import org.apache.iceberg.catalog.Catalog;
import org.apache.iceberg.catalog.TableIdentifier;
import org.apache.iceberg.data.GenericRecord;
import org.apache.iceberg.data.parquet.GenericParquetWriter;
import org.apache.iceberg.hadoop.HadoopCatalog;
import org.apache.iceberg.io.FileAppender;
import org.apache.iceberg.parquet.Parquet;
import org.apache.iceberg.relocated.com.google.common.collect.Lists;
import org.apache.iceberg.types.Types;
import java.io.File;
import java.io.IOException;
import java.time.LocalDate;
import java.time.temporal.ChronoUnit;
import java.util.List;
import static org.apache.iceberg.types.Types.NestedField.optional;
import static org.apache.iceberg.types.Types.NestedField.required;
public class IcebergTableAppend {
public static void main(String[] args) {
System.out.println("Appending records ");
Configuration conf = new Configuration();
String lakehouse = "/tmp/iceberg-test";
conf.set(CatalogProperties.WAREHOUSE_LOCATION, lakehouse);
Schema schema = new Schema(
required(1, "hotel_id", Types.LongType.get()),
optional(2, "hotel_name", Types.StringType.get()),
required(3, "customer_id", Types.LongType.get()),
required(4, "arrival_date", Types.DateType.get()),
required(5, "departure_date", Types.DateType.get()),
required(6, "value", Types.DoubleType.get())
);
PartitionSpec spec = PartitionSpec.builderFor(schema)
.month("arrival_date")
.build();
TableIdentifier id = TableIdentifier.parse("bookings.rome_hotels");
String warehousePath = "file://" + lakehouse;
Catalog catalog = new HadoopCatalog(conf, warehousePath);
// rm -rf /tmp/iceberg-test/bookings
Table table = catalog.createTable(id, schema, spec);
List<GenericRecord> records = Lists.newArrayList();
// generating a bunch of records
for (int j = 1; j <= 12; j++) {
int NUM_ROWS_PER_MONTH = 2300;
for (int i = 0; i < NUM_ROWS_PER_MONTH; i++) {
GenericRecord rec = GenericRecord.create(schema);
rec.setField("hotel_id", (long) (i * 2) + 10000);
rec.setField("hotel_name", "hotel_name-" + i + 1000);
rec.setField("customer_id", (long) (i * 2) + 20000);
rec.setField("arrival_date",
LocalDate.of(2022, j, (i % 23) + 1)
.plus(1, ChronoUnit.DAYS));
rec.setField("departure_date",
LocalDate.of(2022, j, (i % 23) + 5));
rec.setField("value", (double) i * 4.13);
records.add(rec);
}
}
File parquetFile = new File(
lakehouse + "/bookings/rome_hotels/arq_001.parquet");
FileAppender<GenericRecord> appender = null;
try {
appender = Parquet.write(Files.localOutput(parquetFile))
.schema(table.schema())
.createWriterFunc(GenericParquetWriter::buildWriter)
.build();
} catch (IOException e) {
throw new RuntimeException(e);
}
try {
appender.addAll(records);
} finally {
try {
appender.close();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
}
I found out how to fix the Java program.
Just add the lines below to the end of the main method
PartitionKey partitionKey = new PartitionKey(table.spec(), table.schema());
DataFile dataFile = DataFiles.builder(table.spec())
.withPartition(partitionKey)
.withInputFile(localInput(parquetFile))
.withMetrics(appender.metrics())
.withFormat(FileFormat.PARQUET)
.build();
Transaction t = table.newTransaction();
t.newAppend().appendFile(dataFile).commit();
// commit all changes to the table
t.commitTransaction();
Also add to your POM file the dependency below
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>3.3.4</version>
</dependency>
This avoids the runtime error shown below:
java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.lib.input.FileInputFormat
In a Libreoffice Spreadsheet's sheet, I can get access to a Cell, but not to its value. I don't find my error, any feedback is welcome. Here is the code:
import java.io.File;
import java.io.IOException;
import org.jopendocument.dom.spreadsheet.MutableCell;
import org.jopendocument.dom.spreadsheet.Sheet;
import org.jopendocument.dom.spreadsheet.SpreadSheet;
public class MyClass {
protected Sheet dataSheet;
protected File dataCalcFile;
public static void main(String[] args) {
MyClass myClassInstance = new MyClass();
myClassInstance.loadData();
}
public void loadData() {
int numRows=0, numColumnas=0;
MutableCell cell=null;
try {
dataCalcFile = new File(""C:\\temp\\Data.ods"");
dataSheet = SpreadSheet.createFromFile(dataCalcFile).getSheet(0);
numRows = dataSheet.getRowCount();
System.out.println("Number of rows: " + numRows);
System.out.println("Cell at 0,0: " + dataSheet.getCellAt(0, 0));
System.out.println("Nullpointer Exception when getting cell value at 0,0: " + dataSheet.getValueAt(0, 0)); // *** THE INFAMOUS ONE ***
} catch (Exception e) {
System.out.println(e);
}
}
}
And here is the output at console:
Number of rows: 107
Cell at 0,0: <table:table-cell xmlns:table="urn:oasis:names:tc:opendocument:xmlns:table:1.0" table:style-name="ce1" xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" office:value-type="string" xmlns:calcext="urn:org:documentfoundation:names:experimental:calc:xmlns:calcext:1.0" calcext:value-type="string"><text:p xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0">Carpeta</text:p></table:table-cell>
java.lang.NullPointerException
Just found that:
"Yes, LO 7 switched to OpenDocument 1.3. We're working on supporting it. In the mean time, you can change the format to "1.2 extended". Go to Options, then Load/Save, then General, then ODF format version."
It works for me, but am excited to hear my customers opinion...
I found that it must be some incompatibility between the library jopendocument and the newer versions of Libreoffice (7.x), as the code above:
Works as expected if the spreadsheet is edited with Libreoffice 6.4.3.2
Spits the null pointer exception out if the spreadsheet is edited with Libreoffice 7.0.1.2
In some version in between Calc produces an XML not understood by jopendocument.
Instead of using
getValueAt(0, 0),
you can use
getCellAt(0, 0).getElement().getValue()
It works for me.
(note, I've resolved my problem and posted the code at the bottom)
I'm playing around with TensorFlow and the backend processing must take place in Java. I've taken one of the models from the https://developers.google.com/machine-learning/crash-course and saved it with tf.saved_model.save(my_model,"house_price_median_income") (using a docker container). I copied the model off and loaded it into Java (using the 2.0 stuff built from source because I'm on windows).
I can load the model and run it:
try (SavedModelBundle model = SavedModelBundle.load("./house_price_median_income", "serve")) {
try (Session session = model.session()) {
Session.Runner runner = session.runner();
float[][] in = new float[][]{ {2.1518f} } ;
Tensor<?> jack = Tensor.create(in);
runner.feed("serving_default_layer1_input", jack);
float[][] probabilities = runner.fetch("StatefulPartitionedCall").run().get(0).copyTo(new float[1][1]);
for (int i = 0; i < probabilities.length; ++i) {
System.out.println(String.format("-- Input #%d", i));
for (int j = 0; j < probabilities[i].length; ++j) {
System.out.println(String.format("Class %d - %f", i, probabilities[i][j]));
}
}
}
}
The above is hardcoded to an input and output but I want to be able to read the model and provide some information so the end-user can select the input and output, etc.
I can get the inputs and outputs with the python command: saved_model_cli show --dir ./house_price_median_income --all
What I want to do it get the inputs and outputs via Java so my code doesn't need to execute python script to get them. I can get operations via:
Graph graph = model.graph();
Iterator<Operation> itr = graph.operations();
while (itr.hasNext()) {
GraphOperation e = (GraphOperation)itr.next();
System.out.println(e);
And this outputs both the inputs and outputs as "operations" BUT how do I know that it is an input and\or an output? The python tool uses the SignatureDef but that doesn't seem to appear in the TensorFlow 2.0 java stuff at all. Am I missing something obvious or is it just missing from TensforFlow 2.0 Java library?
NOTE, I've sorted my issue with the answer help below. Here is my full bit of code in case somebody would like it in the future. Note this is TF 2.0 and uses the SNAPSHOT mentioned below. I make a few assumptions but it shows how to pull the input and output and then use them to run a model
import org.tensorflow.SavedModelBundle;
import org.tensorflow.Session;
import org.tensorflow.Tensor;
import org.tensorflow.exceptions.TensorFlowException;
import org.tensorflow.Session.Run;
import org.tensorflow.Graph;
import org.tensorflow.Operation;
import org.tensorflow.Output;
import org.tensorflow.GraphOperation;
import org.tensorflow.proto.framework.SignatureDef;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import org.tensorflow.proto.framework.MetaGraphDef;
import java.util.Map;
import org.tensorflow.proto.framework.TensorInfo;
import org.tensorflow.types.TFloat32;
import org.tensorflow.tools.Shape;
import java.nio.FloatBuffer;
import org.tensorflow.tools.buffer.DataBuffers;
import org.tensorflow.tools.ndarray.FloatNdArray;
import org.tensorflow.tools.ndarray.StdArrays;
import org.tensorflow.proto.framework.TensorInfo;
public class v2tensor {
public static void main(String[] args) {
try (SavedModelBundle savedModel = SavedModelBundle.load("./house_price_median_income", "serve")) {
SignatureDef modelInfo = savedModel.metaGraphDef().getSignatureDefMap().get("serving_default");
TensorInfo input1 = null;
TensorInfo output1 = null;
Map<String, TensorInfo> inputs = modelInfo.getInputsMap();
for(Map.Entry<String, TensorInfo> input : inputs.entrySet()) {
if (input1 == null) {
input1 = input.getValue();
System.out.println(input1.getName());
}
System.out.println(input);
}
Map<String, TensorInfo> outputs = modelInfo.getOutputsMap();
for(Map.Entry<String, TensorInfo> output : outputs.entrySet()) {
if (output1 == null) {
output1=output.getValue();
}
System.out.println(output);
}
try (Session session = savedModel.session()) {
Session.Runner runner = session.runner();
FloatNdArray matrix = StdArrays.ndCopyOf(new float[][]{ { 2.1518f } } );
try (Tensor<TFloat32> jack = TFloat32.tensorOf(matrix) ) {
runner.feed(input1.getName(), jack);
try ( Tensor<TFloat32> rezz = runner.fetch(output1.getName()).run().get(0).expect(TFloat32.DTYPE) ) {
TFloat32 data = rezz.data();
data.scalars().forEachIndexed((i, s) -> {
System.out.println(s.getFloat());
} );
}
}
}
} catch (TensorFlowException ex) {
ex.printStackTrace();
}
}
}
What you need to do is to read the SavedModelBundle metadata as a MetaGraphDef, from there you can retrieve input and output names from the SignatureDef, like in Python.
In TF Java 1.* (i.e. the client you are using in your example), the proto definitions are not available out-of-the-box from the tensorflow artifact, you need to add a dependency to org.tensorflow:proto as well and deserialize the result of SavedModelBundle.metaGraphDef() into a MetaGraphDef proto.
In TF Java 2.* (the new client actually only available as snapshots from here), the protos are present right away so you can simply call this line to retrieve the right SignatureDef:
savedModel.metaGraphDef().signatureDefMap.getValue("serving_default")
I have trained custom NER and Relation extraction model and I have checked generating triples with corenlp server but when I'm using OpenIEDemo.java
to generate triples it's generating triples having relations "has" and "have" only but not the relations on which I have trained my Relation Extraction model on.
I'm loading custom NER and Relation Extraction model while running the same script. Here is my OpenIEDemo.java file...
package edu.stanford.nlp.naturalli;
import edu.stanford.nlp.ie.util.RelationTriple;
import edu.stanford.nlp.io.IOUtils;
import edu.stanford.nlp.ling.CoreAnnotations;
import edu.stanford.nlp.pipeline.Annotation;
import edu.stanford.nlp.pipeline.StanfordCoreNLP;
import edu.stanford.nlp.semgraph.SemanticGraph;
import edu.stanford.nlp.semgraph.SemanticGraphCoreAnnotations;
import edu.stanford.nlp.util.CoreMap;
import edu.stanford.nlp.util.PropertiesUtils;
import java.util.Collection;
import java.util.List;
import java.util.Properties;
/**
* A demo illustrating how to call the OpenIE system programmatically.
* You can call this code with:
*
* <pre>
* java -mx1g -cp stanford-openie.jar:stanford-openie-models.jar edu.stanford.nlp.naturalli.OpenIEDemo
* </pre>
*
*/
public class OpenIEDemo {
private OpenIEDemo() {} // static main
public static void main(String[] args) throws Exception {
Properties props = new Properties();
props.setProperty("annotators", "tokenize, ssplit, pos, lemma, depparse, natlog, openie");
props.setProperty("ner.model", "./ner/ner-model.ser.gz");
props.setProperty("sup.relation.model", "./relation_extractor/relation_model_pipeline.ser.ser");
StanfordCoreNLP pipeline = new StanfordCoreNLP(props);
// Annotate an example document.
String text;
if (args.length > 0) {
text = args[0];
} else {
text = "Obama was born in Hawaii. He is our president.";
}
Annotation doc = new Annotation(text);
pipeline.annotate(doc);
// Loop over sentences in the document
int sentNo = 0;
for (CoreMap sentence : doc.get(CoreAnnotations.SentencesAnnotation.class)) {
System.out.println("Sentence #" + ++sentNo + ": " + sentence.get(CoreAnnotations.TextAnnotation.class));
// Print SemanticGraph
System.out.println(sentence.get(SemanticGraphCoreAnnotations.EnhancedDependenciesAnnotation.class).toString(SemanticGraph.OutputFormat.LIST));
// Get the OpenIE triples for the sentence
Collection<RelationTriple> triples = sentence.get(NaturalLogicAnnotations.RelationTriplesAnnotation.class);
// Print the triples
for (RelationTriple triple : triples) {
System.out.println(triple.confidence + "\t" +
triple.subjectLemmaGloss() + "\t" +
triple.relationLemmaGloss() + "\t" +
triple.objectLemmaGloss());
}
// Alternately, to only run e.g., the clause splitter:
List<SentenceFragment> clauses = new OpenIE(props).clausesInSentence(sentence);
for (SentenceFragment clause : clauses) {
System.out.println(clause.parseTree.toString(SemanticGraph.OutputFormat.LIST));
}
System.out.println();
}
}
}
Thanks in advance.
As OpenIE module of stanfordCoreNLP not using custom relation model(don't know why) I can not use custom relation extraction model with this code instead I had to run SanfordCoreNLP pipeline adding path for my custom NER and Relation Extraction model in server.properties file and generate triples. If someone know the reason why OpenIE is not using custom Relation Extraction model please comment, it will be very useful for others.
Here is the code to get list of currently running process in windows.
import com.sun.jna.platform.win32.Kernel32;
import com.sun.jna.platform.win32.Tlhelp32;
import com.sun.jna.platform.win32.WinDef;
import com.sun.jna.platform.win32.WinNT;
import com.sun.jna.win32.W32APIOptions;
import com.sun.jna.Native;
public class ListProcesses {
public static void main(String[] args) {
Kernel32 kernel32 = (Kernel32) Native.loadLibrary(Kernel32.class, W32APIOptions.UNICODE_OPTIONS);
Tlhelp32.PROCESSENTRY32.ByReference processEntry = new Tlhelp32.PROCESSENTRY32.ByReference();
WinNT.HANDLE snapshot = kernel32.CreateToolhelp32Snapshot(Tlhelp32.TH32CS_SNAPPROCESS, new WinDef.DWORD(0));
try {
while (kernel32.Process32Next(snapshot, processEntry)) {
System.out.println(processEntry.th32ProcessID + "\t" + Native.toString(processEntry.szExeFile)+"\t"+processEntry.readField(""));
}
}
finally {
kernel32.CloseHandle(snapshot);
}
}
}
But I am unable to get description of the process/service in output.Kindly provide solution to get process description of each running proceess. Thanks in advance.
Instead of loading and invoking Kernel32 you could simply use the following code snippet in windows which uses the Runtime to execute a native process:
public List<String> execCommand(String ... command)
{
try
{
// execute the desired command
Process proc = null;
if (command.length > 1)
proc = Runtime.getRuntime().exec(command);
else
proc = Runtime.getRuntime().exec(command[0]);
// process the response
String line = "";
List<String> output = new ArrayList<>();
try (BufferedReader input = new BufferedReader(new InputStreamReader(proc.getInputStream())))
{
while ((line = input.readLine()) != null)
{
output.add(line);
}
}
return output;
}
catch (IOException e)
{
e.printStackTrace();
}
return Collections.<String>emptyList();
}
and then execute the command which invokes the Windows Management Information Command-line:
List<String> output = execCommand("wmic.exe PROCESS where name='"+processName+"'");
processName should contain the name of the running application or exe you try to get information from.
The returned list will then contain the line output of the status information of the running application. The first entry will contain header-information for the respective fields while the following entries will contain information on all matching process names.
Further infos on WMIC:
MSDN
Product documentation
Generating HTML output for WMI
HTH
As I stumbled today across this post here which showcases how to extract the version info from an executable file, your post came back to my mind and so I started a bit of investigation.
This further post states that the process description can only be extracted from the executable file itself so we need to lay our hands on JNA instead of parsing some output from WMIC or TASKLIST. This post further links the MSDN page for VerQueryValue function which provides a C way of extracting the process description. Here especially the 2nd parameter should be of interest as it defines what to return.
With the code mentioned in the first post it is now a bit of converting the C struct typedefs into Java equivalents. I will post the complete code here which at least works for me on Windows 7 64bit:
Maven pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>at.rovo.test</groupId>
<artifactId>JNI_Test</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>JNI_Test</name>
<url>http://maven.apache.org</url>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
</build>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<!-- JNA 3.4
<dependency>
<groupId>net.java.dev.jna</groupId>
<artifactId>jna</artifactId>
<version>3.4.0</version>
</dependency>
<dependency>
<groupId>net.java.dev.jna</groupId>
<artifactId>platform</artifactId>
<version>3.4.0</version>
</dependency>
-->
<!-- JNA 4.0.0 -->
<dependency>
<groupId>net.java.dev.jna</groupId>
<artifactId>jna</artifactId>
<version>4.0.0</version>
</dependency>
<dependency>
<groupId>net.java.dev.jna</groupId>
<artifactId>jna-platform</artifactId>
<version>4.0.0</version>
</dependency>
<!-- -->
</dependencies>
</project>
LangAndCodePage.java - a helper class which maps the extracted hex-values to human readable output of the language and code page information contained within the translation table. The values therefore are taken from this page here:
package at.rovo.test.jni_test;
import java.util.HashMap;
import java.util.Map;
public final class LangAndCodePage
{
private final static Map<String, String> languages = new HashMap<>();
private final static Map<String, String> codePage = new HashMap<>();
static
{
languages.put("0000", "Language Neutral");
languages.put("0401", "Arabic");
languages.put("0402", "Bulgarian");
languages.put("0403", "Catalan");
languages.put("0404", "Traditional Chinese");
languages.put("0405", "Czech");
languages.put("0406", "Danish");
languages.put("0407", "German");
languages.put("0408", "Greek");
languages.put("0409", "U.S. English");
languages.put("040A", "Castilian Spanish");
languages.put("040B", "Finnish");
languages.put("040C", "French");
languages.put("040D", "Hebrew");
languages.put("040E", "Hungarian");
languages.put("040F", "Icelandic");
languages.put("0410", "Italian");
languages.put("0411", "Japanese");
languages.put("0412", "Korean");
languages.put("0413", "Dutch");
languages.put("0414", "Norwegian ? Bokmal");
languages.put("0810", "Swiss Italian");
languages.put("0813", "Belgian Dutch");
languages.put("0814", "Norwegian ? Nynorsk");
languages.put("0415", "Polish");
languages.put("0416", "Portuguese (Brazil)");
languages.put("0417", "Rhaeto-Romanic");
languages.put("0418", "Romanian");
languages.put("0419", "Russian");
languages.put("041A", "Croato-Serbian (Latin)");
languages.put("041B", "Slovak");
languages.put("041C", "Albanian");
languages.put("041D", "Swedish");
languages.put("041E", "Thai");
languages.put("041F", "Turkish");
languages.put("0420", "Urdu");
languages.put("0421", "Bahasa");
languages.put("0804", "Simplified Chinese");
languages.put("0807", "Swiss German");
languages.put("0809", "U.K. English");
languages.put("080A", "Spanish (Mexico)");
languages.put("080C", "Belgian French");
languages.put("0C0C", "Canadian French");
languages.put("100C", "Swiss French");
languages.put("0816", "Portuguese (Portugal)");
languages.put("081A", "Serbo-Croatian (Cyrillic)");
codePage.put("0000", "7-bit ASCII");
codePage.put("03A4", "Japan (Shift ? JIS X-0208)");
codePage.put("03B5", "Korea (Shift ? KSC 5601)");
codePage.put("03B6", "Taiwan (Big5)");
codePage.put("04B0", "Unicode");
codePage.put("04E2", "Latin-2 (Eastern European)");
codePage.put("04E3", "Cyrillic");
codePage.put("04E4", "Multilingual");
codePage.put("04E5", "Greek");
codePage.put("04E6", "Turkish");
codePage.put("04E7", "Hebrew");
codePage.put("04E8", "Arabic");
}
// prohibit instantiation
private LangAndCodePage()
{
}
public static void printTranslationInfo(String lang, String cp)
{
StringBuilder builder = new StringBuilder();
builder.append("Language: ");
builder.append(languages.get(lang));
builder.append(" (");
builder.append(lang);
builder.append("); ");
builder.append("CodePage: ");
builder.append(codePage.get(cp));
builder.append(" (");
builder.append(cp);
builder.append(");");
System.out.println(builder.toString());
}
}
Last but not least the code which extracts the file version and the file description of the Windows explorer. The code contains plenty of documentation as I used it to learn the stuff myself ;)
package at.rovo.test.jni_test;
import java.io.IOException;
import com.sun.jna.Memory;
import com.sun.jna.Pointer;
import com.sun.jna.platform.win32.VerRsrc.VS_FIXEDFILEINFO;
import com.sun.jna.platform.win32.Version;
import com.sun.jna.ptr.IntByReference;
import com.sun.jna.ptr.PointerByReference;
import java.math.BigInteger;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
public class FileVersion
{
// The structure as implemented by the MSDN article
public static class LANGANDCODEPAGE extends Structure
{
/** The language contained in the translation table **/
public short wLanguage;
/** The code page contained in the translation table **/
public short wCodePage;
public LANGANDCODEPAGE(Pointer p)
{
useMemory(p);
}
public LANGANDCODEPAGE(Pointer p, int offset)
{
useMemory(p, offset);
}
public static int sizeOf()
{
return 4;
}
// newer versions of JNA require a field order to be set
#Override
protected List getFieldOrder()
{
List fieldOrder = new ArrayList();
fieldOrder.add("wLanguage");
fieldOrder.add("wCodePage");
return fieldOrder;
}
}
public static void main(String[] args) throws IOException
{
// http://msdn.microsoft.com/en-us/library/ms647464%28v=vs.85%29.aspx
//
// VerQueryValue will take two input and two output parameters
// 1. parameter: is a pointer to the version-information returned
// by GetFileVersionInfo
// 2. parameter: will take a string and return an output depending on
// the string:
// "\\"
// Is the root block and retrieves a VS_FIXEDFILEINFO struct
// "\\VarFileInfo\Translation"
// will return an array of Var variable information structure
// holding the language and code page identifier
// "\\StringFileInfo\\{lang-codepage}\\string-name"
// will return a string value of the language and code page
// requested. {lang-codepage} is a concatenation of a language
// and the codepage identifier pair found within the translation
// array in a hexadecimal string! string-name must be one of the
// following values:
// Comments, InternalName, ProductName, CompanyName,
// LegalCopyright, ProductVersion, FileDescription,
// LegalTrademarks, PrivateBuild, FileVersion,
// OriginalFilename, SpecialBuild
// 3. parameter: contains the address of a pointer to the requested
// version information in the buffer of the 1st parameter.
// 4. parameter: contains a pointer to the size of the requested data
// pointed to by the 3rd parameter. The length depends on
// the input of the 2nd parameter:
// *) For root block, the size in bytes of the structure
// *) For translation array values, the size in bytes of
// the array stored at lplpBuffer;
// *) For version information values, the length in
// character of the string stored at lplpBuffer;
String filePath = "C:\\Windows\\explorer.exe";
IntByReference dwDummy = new IntByReference();
dwDummy.setValue(0);
int versionlength =
Version.INSTANCE.GetFileVersionInfoSize(filePath, dwDummy);
if (versionlength > 0)
{
// will hold the bytes of the FileVersionInfo struct
byte[] bufferarray = new byte[versionlength];
// allocates space on the heap (== malloc in C/C++)
Pointer lpData = new Memory(bufferarray.length);
// will contain the address of a pointer to the requested version
// information
PointerByReference lplpBuffer = new PointerByReference();
// will contain a pointer to the size of the requested data pointed
// to by lplpBuffer.
IntByReference puLen = new IntByReference();
// reads versionLength bytes from the executable file into the FileVersionInfo struct buffer
boolean fileInfoResult =
Version.INSTANCE.GetFileVersionInfo(
filePath, 0, versionlength, lpData);
// retrieve file description for language and code page "i"
boolean verQueryVal =
Version.INSTANCE.VerQueryValue(
lpData, "\\", lplpBuffer, puLen);
// contains version information for a file. This information is
// language and code page independent
VS_FIXEDFILEINFO lplpBufStructure =
new VS_FIXEDFILEINFO(lplpBuffer.getValue());
lplpBufStructure.read();
int v1 = (lplpBufStructure.dwFileVersionMS).intValue() >> 16;
int v2 = (lplpBufStructure.dwFileVersionMS).intValue() & 0xffff;
int v3 = (lplpBufStructure.dwFileVersionLS).intValue() >> 16;
int v4 = (lplpBufStructure.dwFileVersionLS).intValue() & 0xffff;
System.out.println(
String.valueOf(v1) + "." +
String.valueOf(v2) + "." +
String.valueOf(v3) + "." +
String.valueOf(v4));
// creates a (reference) pointer
PointerByReference lpTranslate = new PointerByReference();
IntByReference cbTranslate = new IntByReference();
// Read the list of languages and code pages
verQueryVal = Version.INSTANCE.VerQueryValue(
lpData, "\\VarFileInfo\\Translation", lpTranslate, cbTranslate);
if (cbTranslate.getValue() > 0)
{
System.out.println("Found "+(cbTranslate.getValue()/4)
+ " translation(s) (length of cbTranslate: "
+ cbTranslate.getValue()+" bytes)");
}
else
{
System.err.println("No translation found!");
return;
}
// Read the file description
// msdn has this example here:
// for( i=0; i < (cbTranslate/sizeof(struct LANGANDCODEPAGE)); i++ )
// where LANGANDCODEPAGE is a struct holding two WORDS. A word is
// 16 bits (2x 8 bit = 2 bytes) long and as the struct contains two
// words the length of the struct should be 4 bytes long
for (int i=0; i < (cbTranslate.getValue()/LANGANDCODEPAGE.sizeOf()); i++))
{
// writes formatted data to the specified string
// out: pszDest - destination buffer which receives the formatted, null-terminated string created from pszFormat
// in: ccDest - the size of the destination buffer, in characters. This value must be sufficiently large to accomodate the final formatted string plus 1 to account for the terminating null character.
// in: pszFormat - the format string. This string must be null-terminated
// in: ... The arguments to be inserted into the pszFormat string
// hr = StringCchPrintf(SubBlock, 50,
// TEXT("\\StringFileInfo\\%04x%04x\\FileDescription"),
// lpTranslate[i].wLanguage,
// lpTranslate[i].wCodePage);
// fill the structure with the appropriate values
LANGANDCODEPAGE langCodePage =
new LANGANDCODEPAGE(lpTranslate.getValue(), i*LANGANDCODEPAGE.sizeOf());
langCodePage.read();
// convert short values to hex-string:
// https://stackoverflow.com/questions/923863/converting-a-string-to-hexadecimal-in-java
String lang = String.format("%04x", langCodePage.wLanguage);
String codePage = String.format("%04x",langCodePage.wCodePage);
// see http://msdn.microsoft.com/en-us/library/windows/desktop/aa381058.aspx
// for proper values for lang and codePage
LangAndCodePage.printTranslationInfo(lang.toUpperCase(), codePage.toUpperCase());
// build the string for querying the file description stored in
// the executable file
StringBuilder subBlock = new StringBuilder();
subBlock.append("\\StringFileInfo\\");
subBlock.append(lang);
subBlock.append(codePage);
subBlock.append("\\FileDescription");
printDescription(lpData, subBlock.toString());
}
}
else
System.out.println("No version info available");
}
private static void printDescription(Pointer lpData, String subBlock)
{
PointerByReference lpBuffer = new PointerByReference();
IntByReference dwBytes = new IntByReference();
// Retrieve file description for language and code page "i"
boolean verQueryVal = Version.INSTANCE.VerQueryValue(
lpData, subBlock, lpBuffer, dwBytes);
// a single character is represented by 2 bytes!
// the last character is the terminating "\n"
byte[] description =
lpBuffer.getValue().getByteArray(0, (dwBytes.getValue()-1)*2);
System.out.println("File-Description: \""
+ new String(description, StandardCharsets.UTF_16LE)+"\"");
}
}
Finally, the output I'm receiving on my German Windows 7 64bit:
[exec:exec]
Version: 6.1.7601.17567
Found 1 translation(s) (length of cbTranslate: 4 bytes)
Language: German (0407); CodePage: Unicode (04B0);
File-Description: "Windows-Explorer"
HTH
#Edit: updated the code to use a class representation of the struct to simplify the extraction of the values (dealing with bytes does require them to swith the order of the bytes received - big and little endian issue)
Found after some tries an application that uses multiple languages and code pages within their file where I could test the output of multiple translations.
#Edit2: refactored the code and put it up on github. As mentioned in the comment, the output running the code from the github repo for the Logitech WingMan Event Monitor returns multiple language and codepage segments:
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXX Information contained in: C:\Program Files\Logitech\Gaming Software\LWEMon.exe
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
File: C:\Program Files\Logitech\Gaming Software\LWEMon.exe
Version: 5.10.127.0
Language: U.S. English
CodePage: Multilingual
Original-Filename: LWEMon.exe
Company-Name: Logitech Inc.
File-Description: Logitech WingMan Event Monitor
File-Version: 5.10.127
Product-Version: 5.10.127
Product-Name: Logitech Gaming Software
Internal-Name: LWEMon
Private-Build:
Special-Build:
Legal-Copyright: © 1999-2010 Logitech. All rights reserved.
Legal-Trademark: Logitech, the Logitech logo, and other Logitech marks are owned by Logitech and may be registered. All other trademarks are the property of their respective owners.
Comment: Created by the WingMan Team.
File: C:\Program Files\Logitech\Gaming Software\LWEMon.exe
Version: 5.10.127.0
Language: Japanese
CodePage: Multilingual
Original-Filename: LWEMon.exe
Company-Name: Logicool Co. Ltd.
File-Description: Logicool WingMan Event Monitor
File-Version: 5.10.127
Product-Version: 5.10.127
Product-Name: Logicool Gaming Software
Internal-Name: LWEMon
Private-Build:
Special-Build:
Legal-Copyright: © 1999-2010 Logicool Co. Ltd. All rights reserved.
Legal-Trademark: Logicool, the Logicool logo, and other Logicool marks are owned by Logicool and may be registered. All other trademarks are the property of their respective owners.
Comment: Created by the WingMan Team.
(Answered by the OP in an answer-response. Converted to a community wiki answer. See Question with no answers, but issue solved in the comments (or extended in chat) )
Actually I find out another way ie. Windows PowerShell commands:
get-process notepad | select-object description.
Therefore, I am using command line to get descriptions of currently running process. Similarly for Services:
get-service | where {$_.status -eq 'running'}.