We Keep Coding

Run Java Jar within Python Google Cloud Function

I'm trying to port over some old Ruby code I used to run on Heroku to a Python-based Google Cloud Function.
This code runs Apple's Reporter tool which is "a command-line tool that you can use to download your Sales and Trends reports and Payments and Financial Reports". Docs can be found here.
The Ruby code worked well for years until yesterday, running on Heroku with a Ruby + Java build pack. A small snippet of this, where options are args received :
options = [
vendor_id,
file_type,
sub_file_type,
'Daily',
trimmed_date,
version
]
Dir.chdir("#{Rails.root}/tmp/") do
stdout, stderr, status = Open3.capture3("java -jar #{Rails.root}/public/jars/Reporter.jar p=Reporter.properties m=Robot.XML Sales.getReport #{options.join(', ')}")
return {:status => status, :error => stderr.to_s, :stdout => stdout.to_s }
end
The error I'm seeing on Heroku after no code or stack updates is Network is available but cannot connect to application. Check your proxy and firewall settings and try again.
Most of our other similar processes have been moved to Google Cloud Functions, so after getting nowhere with the above error I thought I'd move this also.
So a similar snippet this time in Python:
from subprocess import Popen, PIPE
def execute_reporter_jar(vendor_id, trimmed_date, file_type, api_version):
process = Popen(["java -jar Reporter.jar p=Reporter.properties Sales.getReportVersion Sales, Detailed"], stdin=PIPE, stdout=PIPE, shell=True)
out, err = process.communicate()
print("returncode = %s", process.returncode)
print("stdout = %s", out)
print("stderr = %s", err)
This works well locally, but when I deploy to Gooogle Cloud it seemingly runs successfully in a few ms, however, nothing actually happens and when I dig deeper it seems the subprocess is returning a 127 - command not found error. So it seems the cloud function can't access Java.
After a good 24hrs, I've hit a wall with this. Can anyone help? I have zero Java knowledge and I know cloud functions have a Java runtime, but I would prefer to stick with Python.
The ultimate aim is for Apple's reporter to run and save the requested file to Google Cloud Storage.
Thanks in advance!

The execution environment for Cloud Function's with Python runtime (both 3.7 and 3.8) is currently based in Ubuntu 18.04 (check the information in this link).
The runtime only includes the following system packages and running subprocess is usually not a recommended idea as the system packages included are limited.
If it's paramount for you to stick with Python you could try to deploy your function using the BuildPack CLI and extending the builder image to install Java on the Python runtime or if your application can be dockerized consider building an image yourself with Java included and deploying your application in Cloud Run.

Matlab installation (LD_LIBRARY_PATH) messes up other library files

I am trying to install Matlab on a Linux machine, but setting LD_LIBRARY_PATH (as the installation requires) breaks other library files. I am not an Linux expert, but I have tried several things and cannot get it working correctly. I have even contacted Matlab support, got the issue elevated to the dev team, and was basically told "haha sucks to suck". I have seen a few other people online have had the same issue, but either their questions were never answered or they had a slightly different problem and their solution didn't apply to me.
Installing on a VM running Ubuntu:
I set LD_LIBRARY_PATH as the instructions say, then it breaks network files. I can ping google.com, but I cannot nslookup google.com or visit it in a browser. Nslookup provides this error:
nslookup: /usr/local/MATLAB/MATLAB_Runtime/v90/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/libdns.so.100)
03-Feb-2016 11:32:22.361 ENGINE_by_id failed (crypto failure)
03-Feb-2016 11:32:22.362 error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:
03-Feb-2016 11:32:22.363 error:260B6084:engine routines:DYNAMIC_LOAD:dso not found:eng_dyn.c:447:
03-Feb-2016 11:32:22.363 error:2606A074:engine routines:ENGINE_by_id:no such engine:eng_list.c:418:id=gost
(null): dst_lib_init: crypto failure
The installation worked though (I can run my Java programs that reference compiled Matlab functions). Unsetting LD_LIBRARY_PATH fixes the network files but then I can't run programs anymore.
Installing on EC2 instance:
On an EC2 instance it does not break the network files (nslookup is fine). Instead it messes up Python library files. Trying to use any aws cli command, I get the error:
File "/usr/bin/aws", line 19, in <module>
import awscli.clidriver
File "/usr/lib/python2.7/dist-packages/awscli/clidriver.py", line 16, in <module>
import botocore.session
File "/usr/lib/python2.7/dist-packages/botocore/session.py", line 25, in <module>
import botocore.config
File "/usr/lib/python2.7/dist-packages/botocore/config.py", line 18, in <module>
from botocore.compat import six
File "/usr/lib/python2.7/dist-packages/botocore/compat.py", line 139, in <module>
import xml.etree.cElementTree
File "/usr/lib64/python2.7/xml/etree/cElementTree.py", line 3, in <module>
from _elementtree import *
ImportError: PyCapsule_Import could not import module "pyexpat"
Printing sys.path in Python shows lib-dynload is already there though, so it doesn't seem to the problem.
And when trying to run the program, I get:
Exception in thread "main" java.lang.LinkageError: libXt.so.6: cannot open shared object file: No such file or directory
at com.mathworks.toolbox.javabuilder.internal.DynamicLibraryUtils.dlopen(Native Method)
at com.mathworks.toolbox.javabuilder.internal.DynamicLibraryUtils.loadLibraryAndBindNativeMethods(DynamicLibraryUtils.java:134)
at com.mathworks.toolbox.javabuilder.internal.MWMCR.<clinit>(MWMCR.java:1529)
at VectorAddExample.VectorAddExampleMCRFactory.newInstance(VectorAddExampleMCRFactory.java:48)
at VectorAddExample.VectorAddExampleMCRFactory.newInstance(VectorAddExampleMCRFactory.java:59)
at VectorAddExample.VectorAddClass.<init>(VectorAddClass.java:62)
at com.mypackage.Example.main(Example.java:13)
I'm at a brick wall and really have no clue how to proceed.

Maybe something else already needs LD_LIBRARY_PATH set to work. Make sure you prepend not overwrite:
export LD_LIBRARY_PATH=new/path:$LD_LIBRARY_PATH
Edit:
OK, if LD_LIBRARY_PATH was initially empty, this suggests that Matlab comes with shared libraries that are incompatible with your system ones:
nslookup: /usr/local/MATLAB/MATLAB_Runtime/v90/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/libdns.so.100)
suggests that /usr/lib/libdns.so.100 needs libcrypto.so.1.0.0, which is now being resolved to the one that comes with MATLAB, which is incompatible.
You can check the dependencies of a dll by
ldd /usr/lib/libcrypto.so.1.0.0
and hopefully you can find a configuration that keeps both MATLAB and your system happy. Unfortunately, this may involve a lot of trial and error.
If there is no such configuration, you can try setting LD_LIBRARY_PATH only when you run MATLAB:
LD_LIBRARY_PATH=$MATLAB_LD_LIBRARY_PATH matlab
Edit 2:
Well, for the Python issue, it seems to boil down to pyexpat, which is a wrapper around the standard expat XML parser. Try doing (name guessed since I don't have a Linux right now):
ldd /usr/local/lib/python2.7/site-packages/libpyexpat.so
and see what that depends on. Probably, it will be libexpat.so, which is now being resolved to MATLAB's version.

try the following command:
export LD_LIBRARY_PATH=/usr/local/MATLAB/MATLAB_Runtime/v90/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v90/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v90/sys/os‌​/glnxa64:$LD_LIBRARY_PATH

Perhaps not helpful for OP but if you are generating a python package with MATLAB, you could modify the generated __init__.py file MATLAB creates for your package.
Specifically, the generated __init__.py file contains the following line (as of MATLAB 2017a):
PLATFORM_DICT = {'Windows': ['PATH','dll',''], 'Linux': ['LD_LIBRARY_PATH','so','libmw'], 'Darwin': ['DYMCR_LIBRARY_PATH','dylib','libmw']}
For Linux platform, you could simply replace LD_LIBRARY_PATH with something else such as MCR_LIBRARY_PATH to prevent mucking with your shared libs.
sed -i -e 's/LD_LIBRARY_PATH/MCR_LIBRARY_PATH/g' /MY/PACKAGE/BUILD/PATH/__init__.py
Then obviously export MCR_LIBRARY_PATH before using python.

Why couldn't "org.antlr.v4.runetime.misc.TestRig" not be found or load?

So, here's my problem. I've got my ANTLR4 code successfully compiled, without errors and now I want to test it out. The ANTLR4 Documentation tells me, to test my applications, I shall do this:
java org.antlr.v4.runtime.misc.TestRig
I've tried this and got following error:
Error: Main Class org.antlr.v4.runtime.misc.TestRig couldn't be found or load.
I've checked if my CLASSPATH wasn't set, but everything was correctly set as it should be. I also tried moving the file directly to my test folder and opened CMD there and tried it again, I occur the same error. Searching in the Internet didn't help, as no one seemed to have occurred this error with ANTLR4 before.
Specs:
Java 1.7.0.55
ANTLR 4.4

There seems to be something wrong with your classpath, contrary to your belief everything is okay.
When I download the ANTLR 4 JAR and run TestRig:
wget http://www.antlr.org/download/antlr-4.4-complete.jar
...
java -cp antlr-4.4-complete.jar org.antlr.v4.runtime.misc.TestRig
I see the following on my console:
java org.antlr.v4.runtime.misc.TestRig GrammarName startRuleName
[-tokens] [-tree] [-gui] [-ps file.ps] [-encoding encodingname]
[-trace] [-diagnostics] [-SLL]
[input-filename(s)]
Use startRuleName='tokens' if GrammarName is a lexer grammar.
Omitting input-filename makes rig read from stdin.

How to use Compass through JRuby, for embedding in my soft'?

I've to "compile" compass files dynamically in my java soft'. So, following explications found on stackoverflow, I've done these steps :
Get JRuby (1.7.3), configure PATH to jruby.jar and binaries (gem, rake ...) and learn to launch ruby script from Java
Get compass and dependancies gems into a local directory, using following command line
gem install -i ./compass compass
=> So I get a directory ./compass/ containing bin, cache,doc, gems & specifications directories.
Put gems in a jar using following command line (think to use a jar name which is not the same as the gems, to avoid JRuby "require" the jar instead of the gems)
jar cf compass-lib.jar -C compass .
Write a test script test.rb in local directory to test my jar. It contains :
require 'compass'
Compass.add_configuration(
{
:project_path => '.',
:sass_path => 'styles/scss',
:css_path => 'styles/css'
},
'custom' # A name for the configuration, can be anything you want
)
Compass.compiler.compile('test.scss', 'test.css')
Execute this script with JRuby, with following command line :
jruby -I . -rcompass-lib.jar test.rb
And get this error ...
Unable to load Sass. Please install it with one of the following commands:
gem install sass --pre
LoadError: no such file to load -- jar:file:C:/jruby-1.7.3/lib/jruby.jar!/gems/s
ass-3.2.8/lib/sass/../sass
require at org/jruby/RubyKernel.java:1027
require at c:/jruby-1.7.3/lib/ruby/shared/rubygems/custom_require.rb:36
(root) at jar:file:C:/jruby-1.7.3/lib/jruby.jar!/gems/sass-3.2.8/lib/sass/ver
sion.rb:5
require at org/jruby/RubyKernel.java:1027
(root) at c:/jruby-1.7.3/lib/ruby/shared/rubygems/custom_require.rb:1
require at c:/jruby-1.7.3/lib/ruby/shared/rubygems/custom_require.rb:36
require at org/jruby/RubyKernel.java:1027
(root) at jar:file:C:/jruby-1.7.3/lib/jruby.jar!/gems/sass-3.2.8/lib/sass.rb:
9
(root) at c:/jruby-1.7.3/lib/ruby/shared/rubygems/custom_require.rb:1
require at org/jruby/RubyKernel.java:1027
require at c:/jruby-1.7.3/lib/ruby/shared/rubygems/custom_require.rb:60
(root) at jar:file:C:/jruby-1.7.3/lib/jruby.jar!/gems/compass-0.12.2/lib/comp
ass/dependencies.rb:6
each at org/jruby/RubyArray.java:1613
(root) at c:/jruby-1.7.3/lib/ruby/shared/rubygems/custom_require.rb:1
require at org/jruby/RubyKernel.java:1027
require at c:/jruby-1.7.3/lib/ruby/shared/rubygems/custom_require.rb:36
(root) at test.rb:1
An idea of the problem and, maybe, of solution ? ;) Thanks by advance.
[UPDATE #1]
After Jörg W Mittag obvious suggestion, the error disapear. Thanks ;) Now, it just don't find the scss file, so it's a "normal problem" ;)
[UPDATE #2]
Having used compass with success in IRB, it's time to transpose the test through Java. So, I wrote some lines in Java to execute my ruby script, using only absolute path to avoid problems in a first time.
InputStream myFile = ISRessources.getFile("C:/Path/To/My/test.rb");
new ScriptingContainer().runScriptlet(myFile, "test.rb");
But ... how to "load" my compass-lib.jar ? What's the equivalent of the -r in JRuby ? I've tried to includ gems into jruby.jar, to add compass-lib.jar in externals jars in java build path, but both don't work, I still have :
LoadError: no such file to load -- compass
require at org/jruby/RubyKernel.java:1027
(root) at test.rb:1
Any idea ?

The error message says:
Unable to load Sass. Please install it with one of the following commands:
gem install sass --pre

inline::java and strawberry perl

I Tried to install this module : Inline::Java with strawberry perl 5.10 i get the below error message :
dmake: Error code 129, while making '..\blib\arch\auto\Inline\Java\JNI\JNI.dll'
dmake.EXE: Error code 255, while making 'subdirs'
PATL/Inline-Java-0.53.tar.gz
C:\strawberry\c\bin\dmake.EXE -- NOT OK
Running make test
Can't test without successful make
Running make install
Make had returned bad status, install seems impossible
Failed during this command:
PATL/Inline-Java-0.53.tar.gz : make NO
where the following environent varialbes are defined as below :
PERL_INLINE_JAVA_J2SDK =C:\jdk1.7.0
PERL_INLINE_JAVA_JNI =1
also all the jdk1.7.0 subdirectories are added to the path environment variable .
Could someone have a hint what could be this error?
and how can i correct it if possible ?

The issue is covered in this FAQ:
Strawberry Perl - Win32 Wiki
Please edit you post title to indicate you want help with an build issue.