This post is related to this one : How to put properly a libgdx application inside swing application? but the question is different.
In this post I explained that I have two JARs : WorldEditor.jar and GameEngine.jar, and I load GameEngine.jar at runtime from WorldEditor.jar.
My problem is with one of the libraries included in GameEngine.jar, namely JRuby.
When I run java -jar GameEngine.jar everything is fine, but when I launche java -jar worldEditor.jar, the instance of JRuby ScriptManager I use returns null when I call ``getEngineByName`. I just can't point ou what is the problem.
By tracing the list of ScriptManagerFactories, I saw that in the good case I have [JRuby, Rhino], and in the bad one I have only [Rhino].
Would someone have an idea of what's going on ?
I don't have much experience with Java's ScriptEngine, but I ran into this issue while answering another question here. I think your problem boils down to classpath order issues.
Using this code:
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;
public class Script {
public static void main(String args[]) throws Exception {
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("jruby");
System.out.println(engine);
}
}
If I run it two different ways, I get two different results:
$ java -cp .:jruby.jar Script
null
$ java -cp jruby.jar:. Script
org.jruby.embed.jsr223.JRubyEngine#30c01f1c
Looking into it a bit, there is a special file in the jar that registers the various scripting containers:
The ScriptEngineManager uses the service provider mechanism described
in the Jar File Specification to obtain instances of all
ScriptEngineFactories available in the current ClassLoader.
My guess is that the JVM doesn't need to load the JRuby jar in the first case, so it hasn't registered the scripting engine. It would only load that jar when it cannot find some class. This means you might be able to force it to work by using some JRuby object before you ever call into the script.
Related
I have several Spark big data applications written in Scala. These applications have their other version written in R.
I also have a web server application written in Java. This is provided as API for the web GUI. The purpose is to enable GUI to execute these applications and choose the version: R or Spark. I managed to call the R code from the Java API and get the result to JSON. But now it seems to be quite complicated to execute the Spark programs.
Until now, I was able to merge one of the Scala .jar file with the Java API with Maven. I do this by placing my Spark program as a local repository in pom.xml so that the Scala code is included in the final .jar package. I also mentioned Scala and breeze library as dependencies in the pom.xml. And when I try to send a request with the API, of course it throws an error saying java.lang.NoClassDefFoundError: org/apache/spark/sql/SparkSession$. By this point, I realize that it was because I haven't mentioned Spark library in Maven dependencies, but then I think I've been doing it wrong, since Spark applications are generally run by executing spark-submit command in terminal.
So now what I'm thinking is putting the Java API .jar and Scala .jar in one folder, and then executing spark-submit from inside of Java API .jar, targeting the Scala .jar. Is this even correct? And how to execute the spark-submit from Java code? Does it have to be using Runtime.exec() as mentioned in here?
SparkLauncher can be used to submit the spark code(written in scala with precomplied jar scala.jar placed at certain location) from the Java Api code.
The saprk documentaion for using SparkLauncher recommends the below way to submit the spark job pro-grammatically from inside Java Applications. Add the below code in your Java Api code.
import org.apache.spark.launcher.SparkAppHandle;
import org.apache.spark.launcher.SparkLauncher;
public class MyLauncher {
public static void main(String[] args) throws Exception {
SparkAppHandle handle = new SparkLauncher()
.setAppResource("/my/scala.jar")
.setMainClass("my.spark.app.Main")
.setMaster("local")
.setConf(SparkLauncher.DRIVER_MEMORY, "2g")
.startApplication();
// Use handle API to monitor / control application.
}
}
I am working on application that has components written in several languages. I am trying to get functionality that works fine in Java working in Jython. There is some native/C++ functionality that Java access via the JNI and is wrapped by SWIG.
Whenever I try to import all of the classes in the project I get errors that PROJECTJNI cannot be linked. Here is my minimum case to produce:
import sys
sys.path.append('PROJECT.jar')
from com.whatever.project import *
Here is the error message when this is executed:
$ jython Bootstrap.py
"my" variable $jythonHome masks earlier declaration in same scope at /usr/bin/jython line 15.
Traceback (most recent call last):
File "Bootstrap.py", line 9, in <module>
from com.whatever.project import *
java.lang.UnsatisfiedLinkError: com.whatever.project.PROJECTJNI.swig_module_init()V
at com.whatever.project.PROJECTJNI.swig_module_init(Native Method)
at com.whatever.project.PROJECTJNI.<clinit>(PROJECTJNI.java:974)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:278)
at org.python.core.Py.loadAndInitClass(Py.java:909)
at org.python.core.Py.findClassInternal(Py.java:844)
at org.python.core.Py.findClass(Py.java:869)
at org.python.core.packagecache.PackageManager.basicDoDir(PackageManager.java:107)
at org.python.core.packagecache.SysPackageManager.doDir(SysPackageManager.java:138)
at org.python.core.PyJavaPackage.fillDir(PyJavaPackage.java:123)
at org.python.core.imp.importAll(imp.java:1051)
at org.python.core.imp.importAll(imp.java:1039)
at org.python.pycode._pyx0.f$0(Bootstrap.py:9)
at org.python.pycode._pyx0.call_function(Bootstrap.py)
at org.python.core.PyTableCode.call(PyTableCode.java:165)
at org.python.core.PyCode.call(PyCode.java:18)
at org.python.core.Py.runCode(Py.java:1275)
at org.python.util.PythonInterpreter.execfile(PythonInterpreter.java:235)
at org.python.util.jython.run(jython.java:247)
at org.python.util.jython.main(jython.java:129)
java.lang.UnsatisfiedLinkError: java.lang.UnsatisfiedLinkError: com.whatever.project.PROJECTJNI.swig_module_init()V
The line 15 thing shows up on any time we invoke jython, we had been ignoring it.
We can get the classes from the Java Project to work by simply loading them one at a time:
com.whatever.project import Class1
com.whatever.project import Class2
...
com.whatever.project import Class50
This is highly impractical because even for short python script we might need a dozen classes. Many of them are exceptions that we are catching which have unique types. So anything that robustly handles errors might need a huge number of classes.
As per the jython documentation I should be able to hide PROJECTJNI so it is not loaded by doing something like this, but I found the docs less than perfectly clear. Here is what I attempted:
import com.whatever.project
__all__ = dir(com.whatever.project)
__all__.remove('PROJECTJNI')
from com.whatever.project import *
But this fails with errors and still clearly trying to load PROJECTJNI.
I also tried to fix the native executable so it could be linked against correctl. I learned another group using JRuby had no issue including everything, so I decided to check the source and the binary. I found void swig_module_init() in the file Project_wrap.cpp that Swig created. It was hidden behind a macro, but it was there, and objdump confirms:
$objdump libPROJECT.so -t |grep PROJECTJNI |grep init
000000000051a900 l O .data 00000000000009d0 _ZZ59Java_com_whatever_project_PROJECTJNI_swig_1module_1initE7methods
0000000000263f66 g F .text 00000000000000d1 Java_com_whatever_project_PROJECTJNI_swig_1module_1init
Am I doing something wrong with any of my troubleshooting steps? Is the a bug in Jython? Is there a simple Python workaround to make it skip loading PROJECTJNI?
Anything that lets me skip linking this or makes this link correctly will be accepted.
On one of the Java classes, the BinaryLoader, there was a method called void load_binary() that is called in a static section of that class:
public class BinaryLoader
{
static
{ load_binaries(); }
public static void load_binaries()
{
// Deep inside here is a call to actually load the shared library. Using
load_shared_library_from_jar("PROJECT");
}
... // more details here
}
Our Java code using the native API called this method as the class was loaded. Our JRuby code called this when the class was 'touched', just using the Constant's name, the name of the class, invoked the static section. To clarify here is a sample from our docs and live Ruby scripts that does this:
# Tells Jruby to load the Java Interopability layer.
require 'java'
# Loads the PROJECT Java tools.
require 'PROJECT.jar'
# Shorten the PROJECT tool names from their obnoxiously long Java name.
module PROJECT
include_package "com.whatever.project"
end
# Let Ruby know the BinaryLoader exists and it will have PROJECT load all the system binaries.
PROJECT::BinaryLoader
Apparently requiring the JarFile is enough to make all the Native symbols and Java classes available in Ruby and the binary isn't loaded until the last line which is just an expressions that resolves to the name of the class and force the static section to run. The actual method call would have looked like: PROJECT::BinaryLoader.load_binaries
In Jython the static section was never invoked even when calling other static methods on the BinaryLoader class. I forced it to call the method manually by importing the minimum of Java and Native symbols to call the method:
# Tells Jython to load the python system Interopability tools.
import sys
# When searching for python symbols, this Java jar should be search also.
sys.path.append('PROJECT.jar')
# Load just enough the have the JVM Know how to load the native PROJECT libraries
from com.whatever.project import PROJECT
from com.whatever.project import BinaryLoader
# Load any native binaries that are required
BinaryLoader.load_binaries()
# After this line is run in any given script then PROJECT can be used.
from com.whatever.project import *
Clearly the native binaries were simply not loaded as had been thought. Anything that loaded these would have been an acceptable solution.
This works around Jython not invoking the static section, by doing the work it should have as early as possible. This seems like a bug in Jython, but it might be that guarantees on static load order are not strongly enforced in Java. Either way because this incompatibility we will likely be removing the static section to prevent future Java ad Ruby Devs from adding something that could fail to load in Python.
I have some code in an Android project that parses HTML using Jsoup. It doesn't use anything Android specific, they're just static methods that take an InputStream, and return my model classes. The app uses Gradle to build itself in Android Studio.
Is there any way I can create a standard Java main method to do something like load HTML from a local file, run it through my parser, and output a JSON file (using Gson on my model class)? I'm thinking maybe I can add a new sourceSet to Gradle like a jvmCompatible set of classes? I would greatly prefer not to copy my code to a separate project.
EDIT:
I guess I didn't make this clear, but I would like the be able to run this locally on my dev machine from the command line, rather than on an Android device or emulator.
You don't necessarily need to do anything in the build file to set this up; the build file generates Java .class files, and you can feed them to Java directly from the command line. You can add a main method to any class:
package com.example.foo;
class MyClass {
...
public static void main(String [] args) {
...
}
}
The main method will be happily ignored util you invoke it via the Java command line. You can do this by setting your classpath to the intermediate build directory and telling the Java command line which class to start:
java -classpath app/build/intermediates/classes/debug/ com.example.foo.MyClass
where you pass in the path to the build/intermediates/classes/debug directory in your app module's build output, and the fully-qualified name of the class.
Note that if you're running a release build that uses ProGuard, this main method could get stripped out if it's not otherwise referenced in the code.
Make sure you don't access any Android classes or you'll get a runtime error.
As an aside, you might find it worthwhile to separate out your Java-only code into a Java-only module in the build. Among other things, it would let you use JUnit to write nice test cases for the classes within; if you're asking this question because you want to do some testing of your parser, you might find it convenient to do so within the auspices of a unit test.
I'm using Grails 2.1.0, and I have a Groovy class that I've written that's not dependent on services, controllers, or any of the other Grails goodness. It uses some .jar libraries and other classes that are already in the Grails classpath.
I want to:
Run the Groovy class (or a Java class, it shouldn't mattter) use the other libraries/classes that Grails already has on its classpath (not services, not controllers, none of that).
Be able to access the command line arguments [this is required]
Does not require bootstrapping the entire Grails environment (I need the classpath obviously, but nothing else)
Ideally, I'd like to be able to do something like this:
java -classpath (I_HAVE_NO_IDEA_HOW_TO_DETERMINE_THIS) com.something.MyClass param1 param2 param3
Things I've already looked into:
Using "grails create-script" which results in a Gant script.
Using "grails run-script"
The first one (using a Gant script) seems horribly wrong to me. Using an Gant script as some sort of intermediary wrapper seems to require bootstrapping the whole Grails environment, plus I have to figure out how to get a reference to the actual class I want to call which seems to be difficult (but I Am Not A Gant Expert, so enlighten me). =)
The second one (using run-script) sort of works... I've used this approach to call service methods before, but it has two problems: first, it bootstraps the entire Grails environment; second, there does not appear to be any way to pass the command-line arguments easily.
Really, I just want the stuff in the classpath (and my command-line parameters) and to be able to call the main() method of my class with minimial frustration. That being said, if you can come up with a working example of any sort that solves the issue (even if it involves some intermediary Gant or other class) I'll be happy to use that approach.
Thanks.
Update: A solution that works with a Gant task, still open to better ideas if anyone has any...
In scripts/FooBar.groovy
includeTargets << grailsScript("_GrailsInit")
target(main: "Runs a generic script and passes parameters") {
def myclass = classLoader.loadClass('com.whatever.scripting.GenericRunScript')
myclass.execute(args);
}
setDefaultTarget(main)
In src/groovy/com/whatever/scripting/GenericRunScript.groovy
package com.whatever.scripting
class GenericRunScript {
public static execute(def args) {
println "args=" + args.inspect()
}
}
Then from the command line, at while in the root directory of the Grails project:
$ grails compile
| Environment set to development.....
| Compiling 2 source files.
$ grails foo-bar test one two
| Environment set to development....
args='test\none\ntwo'
Note 1: When I first did this, I kept forgetting the compile statement, so I added that in.
Note 2: Yes, the args are separated by carriage returns; fixing that is left as an exercise to the reader.
The way described above would work but all grails facility will be gone including domains and dependencies.
If you require everything that you have defined in your grails project, the run-script command will do the trick
grails run-script [path to your groovy file]
http://grails.org/doc/latest/ref/Command%20Line/run-script.html
As described in http://grails.org/doc/latest/guide/commandLine.html, you can include targets _GrailsClasspath and _GrailsArgParsing, and whatever else you need. For example, if you want to parse command-line arguments without creating a second script:
$ grails create-script ArgsScript
| Created file scripts/ArgsScript.groovy
Now edit the script scripts/ArgsScript.groovy as follows:
includeTargets << grailsScript("_GrailsArgParsing") // grailsScript("_GrailsInit")
target(main: "The description of the script goes here!") {
println argsMap
for (p in argsMap['params'])
println p
}
setDefaultTarget(main)
See the result:
$ grails args-script one two three=four
| Environment set to development....
[params:[one, two, three=four]]
one
two
three=four
Update: well, it is not as easy as I thought. Basically, you can either run a script as a Gant task, e.g. by doing grails myscript, or as a script, e.g. by doing grails run-script src/groovy/MyScript.groovy. In the first case you have access to parameters, as I already explained, but you still miss some of the Grails environment, which is, perhaps, a bug. For example, you can't really access scripts or classes defined in src/groovy/ from a Gant task. On the other hand, as was already discussed, if you use run-script, you can't get the arguments.
However, you can use System.getProperty to pass command-line arguments with the -Dproperty=value syntax. Also see Java system properties and environment variables
I'm working on a Java/JRuby project which needs to be able to be able to interact with GAMS. I know we can use the Java API, but I would really like to be able to access it using JRuby if possible, since we're hoping to eventual add a DSL and some other complexity I'm not really excited about having to implement in pure Java.
Following the official Java API documentation for GAMS, I have downloaded and setup everything necessary to run GAMS from the command line, but I can't figure out how to include the GAMS directory in LD_LIBRARY_PATH and still run JRuby irb. When I run
export LD_LIBRARY_PATH=/home/wikk/Downloads/gams24.0_linux_x64_64_sfx
Then try to run irb with JRuby, I get
jruby: /home/wikk/Downloads/gams24.0_linux_x64_64_sfx/libstdc++.so.6: version 'GLIBCXX_3.4.15' not found (required by jruby)
I think this is what the documentation is asking me to do to run a Java program that calls the API, is there maybe some way to set LD_LIBRARY_PATH in irb, but before importing all the Java class files? I can do this successfully if I don't set LD_LIBRARY_PATH, but then GAMS tells me it can't find the main program when I try to create a new GAMSWorkspace object:
irb(main):002:0> ws = GAMSWorkspace.new
Java::ComGamsApi::GAMSException: could not find a GAMS system directory from
your environment variable, please set up properly before running a program!
from com.gams.api.GAMSWorkspace.verifySystemDirectory(GAMSWorkspace.java:335)
Am I doing this wrong? or does the API require some Java feature that isn't implemented in JRuby?
Finally came back to this problem, got it working through some trial and error. I also needed to run jruby with the -J-Djava.library.path=[GAMSDIR]/apifiles/Java/api flag, and add [GAMSDIR]/apifiles/Java/api/GAMSJavaAPI.jar to the classpath.
Once this is all in place, you can run gams models from ruby scripts:
import com.gams.api.GAMSWorkspace
import com.gams.api.GAMSJob
import com.gams.api.GAMSVariable
import com.gams.api.GAMSVariableRecord
import com.gams.api.GAMSWorkspace
ws = GAMSWorkspace.new
j1 = ws.addJobFromGamsLib('trnsport')
j1.run
j1.out_db.get_variable('x').each_entry do |rec|
puts "x(#{rec.get_keys[0]}, #{rec.get_keys[1]}): level = #{rec.get_level}, marginal = #{rec.get_marginal}"
end
I am writing here because it is the only thread related to the GAMS Java API problem.
In Eclipse, you have to go to "Run Configurations" and add two things:
1. (As already said) add a "-Djava.library.path=[GAMSDIR]\apifiles\Java\api\" to VM arguments
2. Go to Environment and SET explicitly a PATH variable to [GAMSDIR]. For some reason seeting path through windows is not working