We Keep Coding

Run Java Jar within Python Google Cloud Function

I'm trying to port over some old Ruby code I used to run on Heroku to a Python-based Google Cloud Function.
This code runs Apple's Reporter tool which is "a command-line tool that you can use to download your Sales and Trends reports and Payments and Financial Reports". Docs can be found here.
The Ruby code worked well for years until yesterday, running on Heroku with a Ruby + Java build pack. A small snippet of this, where options are args received :
options = [
vendor_id,
file_type,
sub_file_type,
'Daily',
trimmed_date,
version
]
Dir.chdir("#{Rails.root}/tmp/") do
stdout, stderr, status = Open3.capture3("java -jar #{Rails.root}/public/jars/Reporter.jar p=Reporter.properties m=Robot.XML Sales.getReport #{options.join(', ')}")
return {:status => status, :error => stderr.to_s, :stdout => stdout.to_s }
end
The error I'm seeing on Heroku after no code or stack updates is Network is available but cannot connect to application. Check your proxy and firewall settings and try again.
Most of our other similar processes have been moved to Google Cloud Functions, so after getting nowhere with the above error I thought I'd move this also.
So a similar snippet this time in Python:
from subprocess import Popen, PIPE
def execute_reporter_jar(vendor_id, trimmed_date, file_type, api_version):
process = Popen(["java -jar Reporter.jar p=Reporter.properties Sales.getReportVersion Sales, Detailed"], stdin=PIPE, stdout=PIPE, shell=True)
out, err = process.communicate()
print("returncode = %s", process.returncode)
print("stdout = %s", out)
print("stderr = %s", err)
This works well locally, but when I deploy to Gooogle Cloud it seemingly runs successfully in a few ms, however, nothing actually happens and when I dig deeper it seems the subprocess is returning a 127 - command not found error. So it seems the cloud function can't access Java.
After a good 24hrs, I've hit a wall with this. Can anyone help? I have zero Java knowledge and I know cloud functions have a Java runtime, but I would prefer to stick with Python.
The ultimate aim is for Apple's reporter to run and save the requested file to Google Cloud Storage.
Thanks in advance!

The execution environment for Cloud Function's with Python runtime (both 3.7 and 3.8) is currently based in Ubuntu 18.04 (check the information in this link).
The runtime only includes the following system packages and running subprocess is usually not a recommended idea as the system packages included are limited.
If it's paramount for you to stick with Python you could try to deploy your function using the BuildPack CLI and extending the builder image to install Java on the Python runtime or if your application can be dockerized consider building an image yourself with Java included and deploying your application in Cloud Run.

Get File Version of .exe in java on Linux

Question - Get File Version of .exe in java on Linux for some strange client.
Solution -
I used JNA library to read file version using Java. Given below code is running fine on windows platform but it is throwing below error on Linux docker image.
"Unable to load library 'version': Error loading shared library libversion.so: No such file or directory Error loading shared library libversion.so: No such file or directory Native library (linux-x86-64/libversion.so) not found in resource path..".
private String GetFileVersion(String filePath) {
File fileToCheck = new File(filePath);
short[] rtnData = new short[4];
int infoSize = Version.INSTANCE.GetFileVersionInfoSize(fileToCheck.getAbsolutePath(), null);
Pointer buffer = Kernel32.INSTANCE.LocalAlloc(WinBase.LMEM_ZEROINIT, infoSize);
try {
Version.INSTANCE.GetFileVersionInfo(fileToCheck.getAbsolutePath(), 0, infoSize, buffer);
IntByReference outputSize = new IntByReference();
PointerByReference pointer = new PointerByReference();
Version.INSTANCE.VerQueryValue(buffer, "\\", pointer, outputSize);
VerRsrc.VS_FIXEDFILEINFO fileInfoStructure = new VerRsrc.VS_FIXEDFILEINFO(pointer.getValue());
rtnData[0] = (short) (fileInfoStructure.dwFileVersionMS.longValue() >> 16);
rtnData[1] = (short) (fileInfoStructure.dwFileVersionMS.longValue() & 0xffff);
rtnData[2] = (short) (fileInfoStructure.dwFileVersionLS.longValue() >> 16);
rtnData[3] = (short) (fileInfoStructure.dwFileVersionLS.longValue() & 0xffff);
return String.format("%s.%s.%s.%s", rtnData[0], rtnData[1], rtnData[2], rtnData[3]);
} catch (Exception exception) {
return null;
} finally {
Kernel32.INSTANCE.GlobalFree(buffer);
}
}

I will start by answering the question that you asked, though I doubt it is what you actually need to know.
The types of different executable file formats are encoded in the first few bytes of the file. For example, ELF files (executables, shared libraries) are described in this Wikipedia page.
So there are a number of ways to find out what kind of executable in Java:
Write some code that reads the first few bytes and decodes the file header information, as per the format described in the Wikipedia link above.
Find an existing Java library that does this and work out how to do this. (Google for "java file magic library" and see what you can find.)
Read about the Linux file command and write some Java code to run file on each library and parse the output.
What I think you actually need to do is a bit different:
Locate the file or files in the file system that the Java is looking for: apparently libversion.so or linux-x86-64/libversion.so. (The file could well be a symlink. Follow it.)
Run file on each file to check that it is the right kind of library. They need to be 32 or 64 bit corresponding the JVM you are running, and the correct ABI and ISA for the platform.
Check that the files are where the JVM expects to find them. The JVM searches for libraries in directories listed in the "java.library.path" system property. You can (if necessary) set the path using a -Djava.library.path=... JVM option.
See "java.library.path – What is it and how to use" for more information on library loading.
(There is absolutely no need to do step 2 "from" or "in" Java.)

I think I have finally worked out what you are doing.
The Version you are using is actually coming from the package com.sun.jna.platform.win32. It is not part of the JNA library (jna.jar). I think it is actually part of jna-platform.jar. If I understand things correctly, that is the generated JNA adapter library for the Windows COM dlls.
If I have that correct, you would actually need the Windows COM native libraries compiled and built for the Linux platform to do what you are trying to do.
AFAIK, that's not possible.
So how could you make this work? Basically you need to do one of the following:
Look for an existing pure Java library for extracting the version information from a Windows ".exe" file. I don't think it is likely that you will find one.
Find the specification for the Windows ".exe" file format and write your own Java code to extract the version information. I haven't looked for the spec to see how much work it would be.
Then you rewrite the code that you added your question to use the alternative API.
The "libversion" file that I mentioned in my other answer is not relevant. It is something else. It is a red herring.

How to use libpostal in my java code?

AddressExpander expander = AddressExpander.getInstance();
ExpanderOptions options = new ExpanderOptions.Builder().build();

Have a read on the section "Installation (Windows)" of the C library documentation in order to install it under windows (https://github.com/openvenues/libpostal#installation-windows); once installed use the JNDI binding in order to make the calls between your java application and libpostal(https://github.com/openvenues/jpostal).
#dunni already answered this for you.

Crash when spawning a JVM inside of a UWP C++ Application

First of: I don't know if this problem is rather related to UWP and it's strange security settings or rather me mis-using JNI's Invocation
Anyhow, I use this code to spawn a jvm inside a c++ static library (With /ZW, so compiling for UWP, but disabling it doesn't change a thing).
The following code is based upon Oracle's Notes and openjdk's code (exeinvoke.c, launcher.c, ...):
void JVM4UWP::loadVM(string vmOptions[], int numOptions, int jni_version) {
/* Copy the VM Options */
options = new JavaVMOption[numOptions];
for (uint8_t i = 0; i < numOptions; i++) {
options[i].optionString = new char[vmOptions[i].length() + 1]; // \0 takes the additional byte
strncpy_s(options[i].optionString, vmOptions[i].length() + 1, vmOptions[i].c_str(), vmOptions[i].length() + 1);
}
//options[0].optionString = "-Djava.class.path=/usr/lib/java";
vm_args = new JavaVMInitArgs();
vm_args->version = jni_version;
vm_args->nOptions = numOptions;
vm_args->options = options;
vm_args->ignoreUnrecognized = false;
/* load and initialize a Java VM, return a JNI interface
* pointer in env */
JNI_CreateJavaVM(&jvm, (void**)&env, vm_args);
delete options;
/* invoke the Main.test method using the JNI */
jclass cls = env->FindClass("Main");
jmethodID mid = env->GetStaticMethodID(cls, "main", "(I)V");
env->CallStaticVoidMethod(cls, mid, 100);
/* We are done. */
jvm->DestroyJavaVM();
}
Now the problem with this is: I added #include <jni.h>, I added lib/jvm.lib as static library (taken from oracle's jdk), however it seems that something is still incomplete.
When I run the application, I get an error upon loadup (App::App of the sample UWP application isn't even run).
The console returns:
Das Programm "[1892] TestJVM.exe" wurde mit Code -1073741515 (0xc0000135) 'Es wurde keine abhängige DLL gefunden' beendet.
Loosely translated it's: Could find no dependant dll.
Now this reminds me of this MSDN Article. I have to admit that I don't fully understand it's contents. Especially since jvm.lib cannot have any Manifests. I tried to add jvm.dll, java.dll and jli.dll to the folder of the .exe but for some reason it still doesn't work.
Since the above code should work without dlls, I think it might be related to UWP, maybe even to my code:
My Setup currently is like this: I have the JVM4UWP static library which in-turn is linked statistically against jvm.lib. This Library is then included into the Demo UWP Project, which I am trying to execute.

Note that only subset of usual Windows APIs is available for uwp applications and I doubt that jvm is compatible with these restrictions. "lib/jvm.lib" might actually be an export library, not static library.
Also you probably might want to get familiar with Using a Win32 DLL in a Universal Windows Platform App.

It looks like JVM library is missing.
Take a look here for a sample where JVM is executed from C code:
http://jnicookbook.owsiak.org/recipe-no-027/
This sample is little bit more complex (it uses POSIX threads) but still, basics are the same.
Make sure that all libraries are on PATH so they can be loaded by either C++ or JVM.
Have fun with JNI!

Okay, so I made progress:
This Question, compiling for Win32 and the fact that you have to embed dll's so they are available, showed me that the jvm.dll was missing, even though I added it to the program's path. Using gflags as pointed out above shows you which dlls it's trying to load and where it tried to locate them.
Actually, you don't have to embed dll's for debugging purposes, but the dlls have to reside inside of the AppX Folder, instead of the Root folder (even though there is also an exe file there).
So since it's not mentioned elsewhere: When linking against jvm.lib, you still need the jvm.dll since this lib only seems to handle dynamic loading/wrapping/etc
Note that only subset of usual Windows APIs is available for uwp applications and I doubt that jvm is compatible with these restrictions.
Yes and no, I had a look at openJDKs Sourcecode and the Majority of it is only File Handling and such, so I guess the part which could require reworking is the Memory and Threading related things.
Anyhow I was expecting such errors rather as Access Violations since some OS imports are simply 0x0 (aka not found), so I could now what I have to change to make it UWP compatible.
I might also give recompiling the openJDK with /ZW a try, but it's a challenging task.
Using a Win32 DLL in a Universal Windows Platform App also tells that at least the static librarys seem to work as is, maybe one needs wrappers for the illegal API to link into the project.

Setting up New Relic for Railo 4.x on Windows

I will like to setup performance monitoring on an application running on Railo 4 using New Relic. I have consulted the java docs on Railo, Railo google groups, etc but no one seems to have a perfect step by step.
Here is what I have done so far:
Extracted newrelic into Railo's install folder.
Added this line to setenv.sh
export JAVA_OPTS="$JAVA_OPTS -javaagent:c/railo/newrelic/newrelic.jar"
Restarted the Railo-Tomcat service
Added this line to the onapplicationstart function
application.NewRelic = createObject( "java", "com.newrelic.api.agent.NewRelic" );
Added this line to the onrequeststart function
if ( structKeyExists( application, "NewRelic" ) ) {
application.NewRelic.setTransactionName( "CFML", CGI.SCRIPT_NAME );
}
My application is still not sending metrics to New Relic. I will appreciate a step by step instruction of what to do as I can't seem to find that anywhere else and I have no idea what to do.

You can't use setenv.sh on Windows. Instead modify the catalina.bat file, or use the Configure Tomcat utility in the Start Menu to set the javaagent option. These steps can be found in more detail in the New Relic documentation

We have more detailed instructions for installing with our supported platforms and frameworks in our documentation. To see a list of the supported frameworks we are compatible with check out https://docs.newrelic.com/docs/java/new-relic-for-java#h2-compatibility
We may be able to work with the Tomcat portion of your environment and you can find helpful installation information at https://docs.newrelic.com/docs/java/java-agent-manual-installation
Should you run into any obstacles, I suggest opening a ticket at http://support.newrelic.com