Lock contention in Groovy Shell interpreter under high load - java

We are evaluating GroovyShell interpreter (v2.4) in our application for dynamically executing standard Java syntax.
Earlier, we were using Java BeanShell Interpreter for it, but it has an issue under high load which prompted us to look for an alternative such as Groovy.
Sample Java Code
static String script = "int y = x * x; System.out.println(\"** value of y ** :: \" + y ); ";
GroovyShell gs = new GroovyShell();
Script evalScript = gs.parse("void evalMethod() {" + script + "}");
// bind variables
Binding binding = new Binding();
binding.setVariable("x", 5);
evalScript.setBinding(binding);
// invoke eval method
evalScript.invokeMethod("evalMethod", null);
We are seeing thread lock contention when multiple threads execute above code simultaneously. We have multiple threads in the blocked state which is degrading application performance.
Blocked Thread Call Stack
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.ClassLoader.loadClass(ClassLoader.java:404)
- waiting to lock <0x00000007bc425e40> (a java.lang.Object)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
- locked <0x00000007bd369ba8> (a groovy.lang.GroovyClassLoader)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:545)
at org.codehaus.groovy.control.ClassNodeResolver.tryAsLoaderClassOrScript(ClassNodeResolver.java:185)
at org.codehaus.groovy.control.ClassNodeResolver.findClassNode(ClassNodeResolver.java:170)
at org.codehaus.groovy.control.ClassNodeResolver.resolveName(ClassNodeResolver.java:126)
at org.codehaus.groovy.control.ResolveVisitor.resolveToOuter(ResolveVisitor.java:676)
at org.codehaus.groovy.control.ResolveVisitor.resolve(ResolveVisitor.java:313)
at org.codehaus.groovy.control.ResolveVisitor.transformPropertyExpression(ResolveVisitor.java:845)
at org.codehaus.groovy.control.ResolveVisitor.transform(ResolveVisitor.java:696)
at org.codehaus.groovy.control.ResolveVisitor.transformMethodCallExpression(ResolveVisitor.java:1081)
at org.codehaus.groovy.control.ResolveVisitor.transform(ResolveVisitor.java:702)
at org.codehaus.groovy.ast.ClassCodeExpressionTransformer.visitExpressionStatement(ClassCodeExpressionTransformer.java:142)
at org.codehaus.groovy.ast.stmt.ExpressionStatement.visit(ExpressionStatement.java:42)
at org.codehaus.groovy.ast.CodeVisitorSupport.visitBlockStatement(CodeVisitorSupport.java:37)
at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitBlockStatement(ClassCodeVisitorSupport.java:166)
at org.codehaus.groovy.control.ResolveVisitor.visitBlockStatement(ResolveVisitor.java:1336)
at org.codehaus.groovy.ast.stmt.BlockStatement.visit(BlockStatement.java:71)
at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitClassCodeContainer(ClassCodeVisitorSupport.java:104)
at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitConstructorOrMethod(ClassCodeVisitorSupport.java:115)
at org.codehaus.groovy.ast.ClassCodeExpressionTransformer.visitConstructorOrMethod(ClassCodeExpressionTransformer.java:53)
at org.codehaus.groovy.control.ResolveVisitor.visitConstructorOrMethod(ResolveVisitor.java:201)
at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitMethod(ClassCodeVisitorSupport.java:126)
at org.codehaus.groovy.ast.ClassNode.visitContents(ClassNode.java:1081)
at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitClass(ClassCodeVisitorSupport.java:53)
at org.codehaus.groovy.control.ResolveVisitor.visitClass(ResolveVisitor.java:1279)
at org.codehaus.groovy.control.ResolveVisitor.startResolving(ResolveVisitor.java:176)
at org.codehaus.groovy.control.CompilationUnit$12.call(CompilationUnit.java:663)
at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:943)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:605)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:554)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
- locked <0x00000007bd372240> (a java.util.HashMap)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
at groovy.lang.GroovyShell.parse(GroovyShell.java:736)
at groovy.lang.GroovyShell.parse(GroovyShell.java:727)
My questions are:
Has anyone experienced such problem? If yes, can you please suggest any possible resolution for it? I googled around for this problem and few blogs suggested to cache groovy Script object. Is Script object thread-safe? Moreover, I need to bind variables to Script object (via groovy Binding object, which is different for each execution in different threads), so I don't think caching groovy Script object is a viable option.
Are there any best practices that I need to follow while using Groovy Shell interpreter with Java?
Any other library that we can consider? My requirement is to dynamically execute standard Java syntax.

groovy parse&compile is a quite heavy part
if your script is static - then you have to parse it just once
if the scripts always different but repeating - you can cache compiled groovy scripts into
ConcurrentHashMap<String, Class<groovy.lang.Script>>
after
Script evalScript = gs.parse(...);
just put evalScript.getClass() into cache of compiled scripts.
and before gs.parse(...) check if you have compiled script and just take a new instance of the cached class if it exists.
check the following examples and execution speed
String script = "int y = x * x; System.out.println(\"** value of y ** :: \" + y ); ";
def ts = System.currentTimeMillis()
for(int i=0;i<100;i++){
GroovyShell gs = new GroovyShell();
Script evalScript = gs.parse("void evalMethod() {" + script + "}");
// bind variables
Binding binding = new Binding();
binding.setVariable("x", i);
evalScript.setBinding(binding);
// invoke eval method
evalScript.invokeMethod("evalMethod", null);
}
println ">> ${System.currentTimeMillis() - ts} millis"
1165 millis
String script = "int y = x * x; System.out.println(\"** value of y ** :: \" + y ); ";
GroovyShell gs = new GroovyShell();
Class<Script> scriptClass = gs.parse("void evalMethod() {" + script + "}").getClass();
def ts = System.currentTimeMillis()
for(int i=0;i<100;i++){
Script evalScript = scriptClass.newInstance();
// bind variables
Binding binding = new Binding();
binding.setVariable("x", i);
evalScript.setBinding(binding);
// invoke eval method
evalScript.invokeMethod("evalMethod", null);
}
println ">> ${System.currentTimeMillis() - ts} millis"
6 millis

Related

Groovy script reloading on runtime

I want to be a able to execute a groovy script from my Java application.
I want to reload the groovy script on runtime if needed. According to their tutorials, I can do something like that :
long now = System.currentTimeMillis();
for(int i = 0; i < 100000; i++) {
try {
GroovyScriptEngine groovyScriptEngine = new GroovyScriptEngine("");
System.out.println(groovyScriptEngine.run("myScript.groovy", new Binding()););
} catch (Exception e) {
e.printStackTrace();
}
}
long end = System.currentTimeMillis();
System.out.println("time " + (end - now));//24 secs
myScript.groovy
"Hello-World"
This works fine and the script is reloaded everytime i change a line in myScript.groovy.
The problem is that this is not time efficient, what it does is parsing the script from the file every time.
Is there any other alternative ? E.g something smarter that checks if the script is already parsed and if it did not change since the last parse do not parse it again.
<< edited due to comments >>
Like mentioned in one of the comments, separating parsing (which is slow) from execution (which is fast) is mandatory if you need performance.
For reactive reloading of the script source we can for example use the java nio watch service:
import groovy.lang.*
import java.nio.file.*
def source = new File('script.groovy')
def shell = new GroovyShell()
def script = shell.parse(source.text)
def watchService = FileSystems.default.newWatchService()
source.canonicalFile.parentFile.toPath().register(watchService, StandardWatchEventKinds.ENTRY_MODIFY)
boolean keepWatching = true
Thread.start {
while (keepWatching) {
def key = watchService.take()
if (key.pollEvents()?.any { it.context() == source.toPath() }) {
script = shell.parse(source.text)
println "source reloaded..."
}
key.reset()
}
}
def binding = new Binding()
def start = System.currentTimeMillis()
for (i=0; i<100; i++) {
script.setBinding(binding)
def result = script.run()
println "script ran: $result"
Thread.sleep(500)
}
def delta = System.currentTimeMillis() - start
println "took ${delta}ms"
keepWatching = false
The above starts a separate watcher thread which uses the java watch service to monitor the parent directory for file modifications and reloads the script source when a modification is detected. This assumes java version 7 or later. The sleep is just there to make it easier to play around with the code and should naturally be removed if measuring performance.
Storing the string System.currentTimeMillis() in script.groovy and running the above code will leave it looping twice a second. Making modifications to script.groovy during the loop results in:
~> groovy solution.groovy
script ran: 1557302307784
script ran: 1557302308316
script ran: 1557302308816
script ran: 1557302309317
script ran: 1557302309817
source reloaded...
script ran: 1557302310318
script ran: 1557302310819
script ran: 1557302310819
source reloaded...
where the source reloaded... lines are printed whenever a change was made to the source file.
I'm not sure about windows but I believe at least on linux that java uses the fsnotify system under the covers which should make the file monitoring part performant.
Should be noted that if we are really unlucky, the script variable will be reset by the watcher thread between the two lines:
script.setBinding(binding)
def result = script.run()
which would break the code as the reloaded script instance would not have the binding set. To fix this, we can for example use a lock:
import groovy.lang.*
import java.nio.file.*
import java.util.concurrent.locks.ReentrantLock
def source = new File('script.groovy')
def shell = new GroovyShell()
def script = shell.parse(source.text)
def watchService = FileSystems.default.newWatchService()
source.canonicalFile.parentFile.toPath().register(watchService, StandardWatchEventKinds.ENTRY_MODIFY)
lock = new ReentrantLock()
boolean keepWatching = true
Thread.start {
while (keepWatching) {
def key = watchService.take()
if (key.pollEvents()?.any { it.context() == source.toPath() }) {
withLock {
script = shell.parse(source.text)
}
println "source reloaded..."
}
key.reset()
}
}
def binding = new Binding()
def start = System.currentTimeMillis()
for (i=0; i<100; i++) {
withLock {
script.setBinding(binding)
def result = script.run()
println "script ran: $result"
}
Thread.sleep(500)
}
def delta = System.currentTimeMillis() - start
println "took ${delta}ms"
keepWatching = false
def withLock(Closure c) {
def result
lock.lock()
try {
result = c()
} finally {
lock.unlock()
}
result
}
which convolutes the code somewhat but keeps us safe against concurrency issues which tend to be hard to track down.

JavaScript ScriptEngine isn't working within Google App Engine for Java (GAE/J)

I am having an issue where I always get a 0 value returned when I try to use the ScriptEngine eval. By using Logger, I was able to determine that there are NullPointerExceptions being generated. After further inspection, it appears that GAE doesn't always return a valid script engine (if ever), because it throws an exception when you try to use it.
My code looks like:
public double myEval(String JsFormulaStr ) {
double solutionValue = 0;
ScriptEngineManager mgr = new ScriptEngineManager();
ScriptEngine eng = mgr.getEngineByName("JavaScript");
if(eng == null) { // Added this block of code to prevent java.lang.NullPointerException...
log.severe("Unable to get Script Engine." );
return 0;
}
try {
Object jsResults = eng.eval(JsFormulaStr);
solutionValue = Double.parseDouble(jsResults.toString());
return solutionValue;
} catch(Exception e) {
log.severe("[ERROR] in getCalculatedSolution_FromJS_ToDouble()::\n\t" +
"Formula String is: " + JsFormulaStr + "\n\t" + e);
return 0;
}
}
Everything works fine if I run it locally as a WebApp (Both in Eclipse & Netbeans. And within Tomcat & Glassfish 4.0).
Some of the strings which I tried to eval:
62.0 / 100
0.0 * 352.0
(0 - 428) * 1000
(0 - 597) * 1000
73.0 / 100
NOTE: The 0's or 0.0's are from other evaluations which have failed in previous calls. Since this function returns 0 on error.
According to Google's JRE Class Whitelist, the ScriptEngineManager and ScriptEngine classes are allowed. So I don't understand why it isn't working as expected.
Any suggestions?
Thanks in advance,
Randy
I've hit the same problem. Although the classes are whitelisted, it seems like their functionality is limited on App Engine. The code works fine on your local machine but fails when deployed to App Engine as there aren't any script engines available (hence the NullPointerException).
Luckily, you can do the same thing using the Rhino engine.
Note: this example builds on that given by Harsha R in https://stackoverflow.com/a/19828128/578821
Download the Rhino Jar and add js.jar to your classpath (you only need js-14.jar if you're using Java 1.4).
/* Example 1: Running a JavaScript function (taken from examples) */
String script = "function abc(x,y) {return x+y;}";
Context context = Context.enter();
try {
ScriptableObject scope = context.initStandardObjects();
Scriptable that = context.newObject(scope);
Function fct = context.compileFunction(scope, script, "script", 1, null);
Object result = fct.call(context, scope, that, new Object[] { 2, 3 });
System.out.println(Context.jsToJava(result, int.class));
}
finally {
Context.exit();
}
/* Example 2: execute a JavaScript statement */
script = "3 + 2 * (4*5)";
context = Context.enter();
try{
Scriptable scope = context.initStandardObjects();
Object result = context.evaluateString(scope, script, "<cmd>", 1, null);
System.out.println(result);
}
finally{
Context.exit();
}

Passing arguments to a lua function with luaj

I'm trying to call a lua function in a Java program using LuaJ. It works fine when I'm not passing any arguments to the closure:
String script = "print 'Hello World!'";
InputStream input = new ByteArrayInputStream(script.getBytes());
Prototype prototype = LuaC.compile(input, "script");
LuaValue globals = JsePlatform.standardGlobals();
LuaClosure closure = new LuaClosure(prototype, globals);
closure.call();
But now I'm trying a lua script with a top-level function that takes an argument and I just can't figure out how to pass in the argument from Java. Here's what I got so far:
String script = "function something(argument)\n"+
"test_string = 'Hello World!'\n"+
"print(test_string)\n"+
"print(argument)\n"+
"end";
InputStream input = new ByteArrayInputStream(script.getBytes());
Prototype prototype = LuaC.compile(input, "script");
LuaValue globals = JsePlatform.standardGlobals();
LuaClosure closure = new LuaClosure(prototype, globals);
closure.invokemethod("something", CoerceJavaToLua.coerce("Foo"));
This results in an Exception on the invokemethod line:
org.luaj.vm2.LuaError: attempt to index ? (a function value)
Thanks for your help!
In lua, the top-level scope is an anonymous function with variable arguments. These are accessed using ... In your example, you don't need the function named something, the chunk itself can be used as an unnamed function.
For example, this code in luaj-3.0-beta1
String script = "argument = ...\n"+
"test_string = 'Hello World!'\n"+
"print(test_string)\n"+
"print(argument)\n";
Globals globals = JsePlatform.standardGlobals();
LuaValue chunk = globals.loadString(script, "myscript");
chunk.call( LuaValue.valueOf("some-arg-value") );
Produced this result for me:
Hello World!
some-arg-value
You can pass in any number of arguments this way.
Since you receive
org.luaj.vm2.LuaError: attempt to index ? (a function value)
as your error; this means that your function is not being created at all.
Try it without \n and give spaces in the variable script. Like this:
String script = "function something(argument) " +
" test_string = 'Hello World!'; " +
" print( test_string ); " +
" print( argument ); " +
" end";

Scala - Fails to execute a process through terminal in a particular scenario

I'm using graphviz to generate graphs based on the messages passed in a scala program.
To invoke the graphviz application from inside the scala program, I'm using the exec() method (similar to Java). It successfully executed the command and created the graph when I used the below code snippet:
var cmd: String = "dot -Tpng Graph.dot -o Graph.png"
var run: Runtime = Runtime.getRuntime() ;
var pr: Process = run.exec(cmd) ;
However It fails to execute after changing the path of the input and output files (I just included a directory inside which the input file and output file resides as shown below)
def main(args: Array[String]): Unit = {
var DirectoryName: String = "Logs"
var GraphFileName: String = DirectoryName + File.separator + "Graph.dot"
val GraphFileObj: File = new File(GraphFileName)
// var cmd: String = "dot -Tpng Graph.dot -o Graph.png"
var cmd: String = "dot -Tpng \"" + GraphFileObj.getAbsolutePath + "\" -o \"" + DirectoryName + File.separator + "Graph.png\"" ;
println(cmd)
var run: Runtime = Runtime.getRuntime() ;
var pr: Process = run.exec(cmd) ;
}
The same command when executed through terminal gives proper output. Can you please help me to find what I'm missing?
exec is not a shell...e.g. quoting won't work as you expect, and thus your path (which may contain spaces, etc) will not be processed as you expect. The command will be broken apart using StringTokenizer, and your literal quotes will be...well..literal.
Use the form of exec that takes an array instead, so you can tokenize the command correctly.
val args = Array[String]("dot", "-Tpng", GraphFileObj.getAbsolutePath, ...);
run.exec(args)

Getting output parameter value set by VBScript (WMI) method in java via JACOB

Am trying to convert a VBScript to java using JACOB - Java COM bridge library.
'Create' method in VBScript accepts a [out] param in it's method and it sets it upon method execution and i couldn't figure out how to retrieve it back via JACOB.
VBScript in question:
Function CreateProcess(strComputer, strCommand)
Dim objWMIService, objProcess
Set objWMIService = GetObject("winmgmts:" & "{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")
Set objProcess = objWMIService.Get("Win32_Process")
errReturn = objProcess.Create (strCommand, Null, Null, intProcessID)
Set objWMIService = Nothing
Set objProcess = Nothing
CreateProcess = intProcessID
End Function
intProcessID is [out] param set after method execution. (Create API contract)
Converted java code(incomplete and modified slightly for demonstration):
public static void createProcess() {
String host = "localhost";
String connectStr = String
.format("winmgmts:{impersonationLevel=impersonate}!\\\\%s\\root\\CIMV2",
host);
ActiveXComponent axWMI = new ActiveXComponent(connectStr);
Variant vCollection = axWMI.invoke("get", new Variant("Win32_Process"));
Dispatch d = vCollection.toDispatch();
Integer processId = null;
int result = Dispatch.call(d, "Create", "notepad.exe", null, null, processId)
.toInt();
System.out.println("Result:" + result);
// WORKS FINE until here i.e. notepad launches properly, however processId still seems to be null. Following commented code is wrong - doesn't work
//Variant v = Dispatch.get(d, "processId"); // even ProcessId doesn't work
//int pId = v.getInt();
//System.out.println("process id:"
// + pId);
// what is the right way to get the process ID set by 'Create' method?
}
Would be great if you could provide some pointers or relevant code. Ask me more if needed. Thanks in advance.
Replacing
Integer processId = null;
with
Variant processId = new Variant(0, true);
should solve the problem. You should then have process ID of the notepad.exe process in the processId variant, and it can be fetched by
processId.getIntRef()

Categories