We Keep Coding

Run Java Jar within Python Google Cloud Function

I'm trying to port over some old Ruby code I used to run on Heroku to a Python-based Google Cloud Function.
This code runs Apple's Reporter tool which is "a command-line tool that you can use to download your Sales and Trends reports and Payments and Financial Reports". Docs can be found here.
The Ruby code worked well for years until yesterday, running on Heroku with a Ruby + Java build pack. A small snippet of this, where options are args received :
options = [
vendor_id,
file_type,
sub_file_type,
'Daily',
trimmed_date,
version
]
Dir.chdir("#{Rails.root}/tmp/") do
stdout, stderr, status = Open3.capture3("java -jar #{Rails.root}/public/jars/Reporter.jar p=Reporter.properties m=Robot.XML Sales.getReport #{options.join(', ')}")
return {:status => status, :error => stderr.to_s, :stdout => stdout.to_s }
end
The error I'm seeing on Heroku after no code or stack updates is Network is available but cannot connect to application. Check your proxy and firewall settings and try again.
Most of our other similar processes have been moved to Google Cloud Functions, so after getting nowhere with the above error I thought I'd move this also.
So a similar snippet this time in Python:
from subprocess import Popen, PIPE
def execute_reporter_jar(vendor_id, trimmed_date, file_type, api_version):
process = Popen(["java -jar Reporter.jar p=Reporter.properties Sales.getReportVersion Sales, Detailed"], stdin=PIPE, stdout=PIPE, shell=True)
out, err = process.communicate()
print("returncode = %s", process.returncode)
print("stdout = %s", out)
print("stderr = %s", err)
This works well locally, but when I deploy to Gooogle Cloud it seemingly runs successfully in a few ms, however, nothing actually happens and when I dig deeper it seems the subprocess is returning a 127 - command not found error. So it seems the cloud function can't access Java.
After a good 24hrs, I've hit a wall with this. Can anyone help? I have zero Java knowledge and I know cloud functions have a Java runtime, but I would prefer to stick with Python.
The ultimate aim is for Apple's reporter to run and save the requested file to Google Cloud Storage.
Thanks in advance!

The execution environment for Cloud Function's with Python runtime (both 3.7 and 3.8) is currently based in Ubuntu 18.04 (check the information in this link).
The runtime only includes the following system packages and running subprocess is usually not a recommended idea as the system packages included are limited.
If it's paramount for you to stick with Python you could try to deploy your function using the BuildPack CLI and extending the builder image to install Java on the Python runtime or if your application can be dockerized consider building an image yourself with Java included and deploying your application in Cloud Run.

Streamsets Error - Bad File Descriptor

I was attempting to use Streamsets to query an Oracle database and publish the data into Kafka. I downloaded Streamsets' tarball on my Mac and unzipped it into my home directory. Running $HOME/streamsets-datacollector-2.1.0.2/bin/streamsets dc started up on my first try, then I followed the instructions here to add the jdbc driver, then the instructions here to configure my streamsets job. However, I got an error: JDBC_00 - Cannot connect to specified database: com.streamsets.pipeline.api.StageException: JDBC_06 - Failed to initialize connection pool: java.sql.SQLRecoverableException: IO Error: Bad file descriptor.

This wound up having something to do with the limit on the number of files a process can have open. When I ran ulimit -n on the laptop, it showed 4864, then I set it to 10,000 via ulimit -n 10000, restarted the streamsets server, and it worked! If I need to keep running this, I will find a more procedural way of setting the ulimit for this process to work around this issue.

you definitely want need increase ulimit -n. To permanently change for all users in Ubuntu try the following:
echo "* soft nofile 60000" > /etc/security/limits.d/*_limits.conf && echo "* hard nofile 60000" >> /etc/security/limits.d/*_limits.conf
substitute 60000 with whatever number you want. I've never had a problem in streamsets using 60k. you should be able to do something similar in bsd.

Why an extra "/.\" in linux path?

Windows Scenario :-
I have a small piece of code which gets me the location of a specific folder in the directory. The code is as given below:
browserPath = this.EnginePath + "\\Chrome_Selenium\\" + "chromedriver.exe";
This gets me the exact path : D:\Engine\Test
I am trying to use the same logic to get the path in linux machine.
The path where the Engine and Chrome Driver is stored in my linux machine (VM) is
/root/Engine/Chrome_Selenium
Now the linux part :-
I am using the following piece of code to get that
browserPath = this.EnginePath + "/Chrome_Selenium/" + "chromedriver";
The path this piece of code fetches is
/root/Engine/.\Chrome_Selenium\chromedriver
Can you please help me understand why the "/.\" is appearing in the path?

I am guessing the reason why ./ is added is because in linux in order to execute a script in the present working directory you need to address it this way: ./script instead of script. Somehow it is attached even in cases where it is not needed (execution in other folder)
As for the main part as Alfe mentioned it does not make any different at the first place.

instance jar file (Snark Client)

On the Snark client website (http://www.staging.threerings.net/snark.html) it says to instance a jar file in order to load the client. I referred to this question on the oracle community website (https://community.oracle.com/message/8935199) which lead to saying you can instance the jar file with:
Snark.main(args);
However when I run my program with that main method call above, it does not display my GUI at all and instead displays this in the console:
The Hunting of the Snark Project - Copyright (C) 2003 Mark J. Wielaard
Snark comes with ABSOLUTELY NO WARRANTY. This is free software, and
you are welcome to redistribute it under certain conditions; read the
COPYING file for details.
snark: Need exactly one <url>, <file> or <dir>.
Usage: snark [--debug [level]] [--no-commands] [--port <port>]
[--share (<ip>|<host>)] (<url>|<file>|<dir>)
--debug Shows some extra info and stacktraces
level How much debug details to show
(defaults to 3, with --debug to 4, highest level is 6).
--no-commands Don't read interactive commands or show usage info.
--port The port to listen on for incomming connections
(if not given defaults to first free port between 6881-6889).
--share Start torrent tracker on <ip> address or <host> name.
<url> URL pointing to .torrent metainfo file to download/share.
<file> Either a local .torrent metainfo file to download
or (with --share) a file to share.
<dir> A directory with files to share (needs --share).
I then referred to this website (http://www.klomp.org/snark/) which gave me an example on how to use the client. The examples given did not make any sense to me and looked like command prompts. This is one of the examples on the website:
./snark somefile.torrent
I have been at this for weeks and any help on how to implement the snark client into my program would be great.

I am not sure what you're passing for args but it seems like the program is not getting any arguments yet it requires some. In order to emulate the website example you show, only programmatically from your own program:
String[] args = { "somefile.torrent" };
Snark.main(args);
If you have a specific file you're wanting to pass in, you might have to use an absolute file path for the string value. The important bit is that main(args) takes a String array that represents command line arguments. So if you were going to run Snark from the command line with snark --debug 6 you'd want args to be {"--debug", "6"}.

jython 2.5.3 on unix : interactive shell with command completion

After having spend 4 days on searching for a working solution, i guess i need to ask.
So far i'm successfully working withj jython 2.5.2 or 2.5.3, with a modifier thinClient.sh that loads what i need. It connects successfully to a DeploymentManager with either IPC or SOAP connector.
However it lacks the readline module:
wsadmin>import readline
WASX7015E: Exception running command: "import readline"; exception information:
com.ibm.bsf.BSFException: exception from Jython:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/usr2/produits/websphere7/AppServer/thinClient/lib/jython/Lib/readline.py", line 20, in <module>
raise ImportError("Cannot access JLineConsole")
ImportError: Cannot access JLineConsole
The goal is to make it interactive, with colored prompt and sofort.
I tried so far with absolutely no success:
org.python.util.JLineConsole / org.python.util.ReadlineConsole (misses readline module)
Readline-1.7 (does nothing)
java-readline / libreadline-java-0.8.0 (misses readline module)
JLine (won't load the jar)
pyreadline (won't integrate to jython)
So:
is it possible with jython 2.5.3 under IBM AIX 64, with a thinClient (jython-installer-2.5.3.jar) to have a real interactive shell with bash like completion and command recall using arrow keys, without having to build/compile something ?
if yes, please somebody describe a working solution:
what's the solution name ?
what's in your wsadmin.properties ?
which libraries to load in LIBPATH ?
which class to load in CLASSPATH ?
which command line to invoque with java ?
There are so many "solutions" or "propositions" to this frenquently asked question on the web, but nowhere did i found a well described or working solution. Too much information scattered all around just becomes a mess :(
thanks for any help !

ok i found a workaround, that was so easy to answer myself:
rlwrap -H $THIN_CLIENT_HOME/logs/rlwrap.history.log -f $THIN_CLIENT_HOME/etc/rlwrap.jython.words.txt -r -pBlue -z $THIN_CLIENT_HOME/etc/rlwrap.prompt.pl $CMDLINE
org.python.util.* and com.ibm.ws.scripting.WasxShell are mutually incompatible