We Keep Coding

Can't use "Handler" approach to adding a URLStreamHandler in AWS Lambda

I'm currently trying to add a URLStreamHandler so I can handle URLs with custom protocols. This works fine when run locally. When deployed to AWS Lambda I get:
java.net.MalformedURLException: unknown protocol: baas
I'm following the "Handler" approach to registering the URLStreamHandler.
I even went as far as copying the code from URL.getURLStreamHandler(String) and added logging into my own code that is run by Lambda:
(Note: this is from the Java 8 source - I realise now that this might not be representative because AWS Lambda uses a Java 11 runtime).
URLStreamHandler handler = null;
String packagePrefixList = null;
packagePrefixList
= java.security.AccessController.doPrivileged(
new sun.security.action.GetPropertyAction(
"java.protocol.handler.pkgs",""));
if (packagePrefixList != "") {
packagePrefixList += "|";
}
// REMIND: decide whether to allow the "null" class prefix
// or not.
packagePrefixList += "sun.net.www.protocol";
LOG.debug("packagePrefixList: " + packagePrefixList);
StringTokenizer packagePrefixIter =
new StringTokenizer(packagePrefixList, "|");
while (handler == null &&
packagePrefixIter.hasMoreTokens()) {
String packagePrefix =
packagePrefixIter.nextToken().trim();
try {
String clsName = packagePrefix + "." + "baas" +
".Handler";
Class<?> cls = null;
LOG.debug("Try " + clsName);
try {
cls = Class.forName(clsName);
} catch (ClassNotFoundException e) {
ClassLoader cl = ClassLoader.getSystemClassLoader();
if (cl != null) {
cls = cl.loadClass(clsName);
}
}
if (cls != null) {
LOG.debug("Instantiate " + clsName);
handler =
(URLStreamHandler)cls.newInstance();
}
} catch (Exception e) {
// any number of exceptions can get thrown here
LOG.debug(e);
}
}
This prints (in Cloudwatch logs):
packagePrefixList: com.elsten.bliss|sun.net.www.protocol (BaasDriver.java:94, thread main)
Try com.elsten.bliss.baas.Handler (BaasDriver.java:108, thread main)
Instantiate com.elsten.bliss.baas.Handler (BaasDriver.java:118, thread main)
com.elsten.bliss.baas.Handler constructor (Handler.java:55, thread main)
So, when run from my own code, in Lambda, it works.
However, the very next line of logging:
java.lang.IllegalArgumentException: URL is malformed: baas://folder: java.lang.RuntimeException
java.lang.RuntimeException: java.lang.IllegalArgumentException: URL is malformed: baas://folder
...
Caused by: java.net.MalformedURLException: unknown protocol: baas
at java.base/java.net.URL.<init>(Unknown Source)
at java.base/java.net.URL.<init>(Unknown Source)
at java.base/java.net.URL.<init>(Unknown Source)
So it seems odd the same code is failing when run in URL. The main difference I can think of is the parent classloader used to load URL and my code are different, and so there's some sort of class loading issue.
The SPI approach can't be used because Lambda doesn't extract META-INF folders!

Initially I thought the old URL.setURLStreamHandlerFactory(URLStreamHandlerFactory) approach was to be avoided, but it turns out this has been improved in recent Java versions, and so I have fallen back to that.
Specifically, a default fallback URLStreamHandlerFactory which is capable of handling streams to http, https, file et al is used as a fallback if the custom one provided cannot handle a stream.
This is a workaround though - it would be interesting to know why the class cannot be loaded.

RServe: control pipe to master process is closed/broken

I have this R script:
palindrome <- function(p) {
for(i in 1:floor(nchar(p)/2) ) {
r <- nchar(p) - i + 1
if ( substr(p, i, i) != substr(p, r, r) ) return(FALSE)
}
return(TRUE)
}
that I am calling from Java using the following code:
connection.serverSource("C:\\Users\\x\\Desktop\\R Script\\Palindrome.R");
the connection is of type RConnection is created as follows:
public void startConnection() {
PATH_TO_R = SystemUtils.IS_OS_UNIX ? "R" : "C:\\Program Files\\R\\R-3.6.1\\bin\\x64\\R.exe";
try {
String cmd = PATH_TO_R + " -e " + "\"library(Rserve);Rserve(port=" + 6311+ ")\"";
Runtime.getRuntime().exec(cmd);
RConnection connection = new RConnection("localhost", 6311);
} catch (IOException | RserveException e) {
e.printStackTrace();
}
}
The problem is I am having this error thrown and have no idea how to fix it:
org.rosuda.REngine.Rserve.RserveException: serverSource failed, request status: control pipe to master process is closed/broken
any help will be much appreciated!

The function serverSource (just like serverEval and serverShutdown) directly works on the main R+Rserve process, this feature is off by default. You can enable it in /etc/Rserv.conf adding the line control enabled (config reference).
If you don't already use an Rserve config file, you can simply create it in the default location /etc/Rserv.conf (linux/macOS) or create it anywhere (also windows) and pass the location as parameter when starting Rserve: R CMD Rserve --RS-conf <your path here> (command line arguments at the bottom of the page).

Calling R script function from Java using rJava

My Requirement -
I need to deploy a Java webservice in a server which internally executes a R scipt file. I googled about various solutions for calling R from Java and the best were rJava and Rserve. Using Rserve I can call R function BUT as I am running this in Windows it can not handle multiple requests at a time and I dont want to switch to Linux.
[Edit]
What I tried -
I have used rJava to call a R function :
String[] args = new String[3];
args[0] = "--quiet"; // Don't print startup message
args[1] = "--no-restore"; // Don't restore anything
args[2] = "--no-save";
String rFilePath = "D:/Dataset_Info/AI-KMS_v2.0/tika/src/main/resources/HSConcordance.R";
Rengine engine = new Rengine(args, false, null);
if (!engine.waitForR()) {
System.out.println("Cannot load R");
}
System.out.print("JRI R-Engine call: ");
engine.eval("source(\"" + rFilePath + "\")");
REXP value = engine.eval("as.integer(a<-simple())");
int a = value.asInt();
System.out.println(a);
Maven dependency -
<dependency>
<groupId>com.github.lucarosellini.rJava</groupId>
<artifactId>JRI</artifactId>
<version>0.9-7</version>
</dependency>
<dependency>
<groupId>com.github.lucarosellini.rJava</groupId>
<artifactId>REngine</artifactId>
<version>0.9-7</version>
</dependency>
<dependency>
<groupId>com.github.lucarosellini.rJava</groupId>
<artifactId>JRIEngine</artifactId>
<version>0.9-7</version>
</dependency>
My R script file -
simple<-function(){
a=1
return(a)
}
Output - JRI R-Engine call: 1
and then it hangs. I debugged it and found that it got stuck in Thread.class
Any kind of help will be greatly appreciated.

The issue was when I am acessing the webservice for the 2nd time it got hanged because we already have an instance of Rengine present which was created at first call.
Rengine re = Rengine.getMainEngine();
if(re == null){
re=new Rengine (new String [] {"--vanilla"}, false, null);
if (!re.waitForR())
{
System.out.println ("Cannot load R");
return "failure";
}
}
re.eval("source(\"" + rFilePath + "\")");
re.eval("copyfile(\""+filePath+"\")");
re.end();
Few points to note -
Check if any instance of Rengine is already present by Rengine re = Rengine.getMainEngine();
Shut down R in the end by re.end();
It may be helpful. thanks.

JavaScript ScriptEngine isn't working within Google App Engine for Java (GAE/J)

I am having an issue where I always get a 0 value returned when I try to use the ScriptEngine eval. By using Logger, I was able to determine that there are NullPointerExceptions being generated. After further inspection, it appears that GAE doesn't always return a valid script engine (if ever), because it throws an exception when you try to use it.
My code looks like:
public double myEval(String JsFormulaStr ) {
double solutionValue = 0;
ScriptEngineManager mgr = new ScriptEngineManager();
ScriptEngine eng = mgr.getEngineByName("JavaScript");
if(eng == null) { // Added this block of code to prevent java.lang.NullPointerException...
log.severe("Unable to get Script Engine." );
return 0;
}
try {
Object jsResults = eng.eval(JsFormulaStr);
solutionValue = Double.parseDouble(jsResults.toString());
return solutionValue;
} catch(Exception e) {
log.severe("[ERROR] in getCalculatedSolution_FromJS_ToDouble()::\n\t" +
"Formula String is: " + JsFormulaStr + "\n\t" + e);
return 0;
}
}
Everything works fine if I run it locally as a WebApp (Both in Eclipse & Netbeans. And within Tomcat & Glassfish 4.0).
Some of the strings which I tried to eval:
62.0 / 100
0.0 * 352.0
(0 - 428) * 1000
(0 - 597) * 1000
73.0 / 100
NOTE: The 0's or 0.0's are from other evaluations which have failed in previous calls. Since this function returns 0 on error.
According to Google's JRE Class Whitelist, the ScriptEngineManager and ScriptEngine classes are allowed. So I don't understand why it isn't working as expected.
Any suggestions?
Thanks in advance,
Randy

I've hit the same problem. Although the classes are whitelisted, it seems like their functionality is limited on App Engine. The code works fine on your local machine but fails when deployed to App Engine as there aren't any script engines available (hence the NullPointerException).
Luckily, you can do the same thing using the Rhino engine.
Note: this example builds on that given by Harsha R in https://stackoverflow.com/a/19828128/578821
Download the Rhino Jar and add js.jar to your classpath (you only need js-14.jar if you're using Java 1.4).
/* Example 1: Running a JavaScript function (taken from examples) */
String script = "function abc(x,y) {return x+y;}";
Context context = Context.enter();
try {
ScriptableObject scope = context.initStandardObjects();
Scriptable that = context.newObject(scope);
Function fct = context.compileFunction(scope, script, "script", 1, null);
Object result = fct.call(context, scope, that, new Object[] { 2, 3 });
System.out.println(Context.jsToJava(result, int.class));
}
finally {
Context.exit();
}
/* Example 2: execute a JavaScript statement */
script = "3 + 2 * (4*5)";
context = Context.enter();
try{
Scriptable scope = context.initStandardObjects();
Object result = context.evaluateString(scope, script, "<cmd>", 1, null);
System.out.println(result);
}
finally{
Context.exit();
}

Mongodb connection issue with java

i am having mongoDB connections issue in java , this is my connection class
public MongoDbUtil() {
try {
System.out.println("1");
String host = "127.0.0.1" ;
String dbName = "m_prod" ;
int port =27017 ;
System.out.println("2");
Mongo m = new Mongo();
System.out.println("3");
ds = new Morphia().createDatastore(m,dbName);
System.out.println("4");
ds.ensureIndexes();
System.out.println("5");
ds.ensureCaps();
System.out.println("1");
} catch(Exception e) {
System.out.println("catch");
}finally{
System.out.println("finally");
System.out.println(ds==null);
} }
only 1 and 2 is printing, after that 'finally' is printing also 'ds' is null, there is no any exception happen ('catch' is not printing)
Mongo server is up and running and i can access from command prompt (Linux) , the Other interesting thing is, its working fine when i call this method by unit test function, but for all other cases above issue happen , what can be the reason ?
Thanks

Mongo() is deprecated, you should use MongoClient() instead - see http://api.mongodb.org/java/2.11.0/com/mongodb/Mongo.html#Mongo()
Still it should find the deprecated constructor. Can you include the imports of your file, please?

If you're using the 3.0 driver, there's a driver-compat layer that will help you transition. You really should use the new API, though.