I am creating a GUI using Jython. I want to program my logic in C. How could I can call a C Function from my Python Code. Sorry if this a newbie question, but I have never worked with linking files except Sparc Assembly from C.
Jython cannot use ctypes, or C extension modules (whether built manually, or with Cython, or otherwise).
The way to do this is the same way as in Java: Through a JNI bridge.
First, you write a C++ wrapper that talks to the so, and uses functions from <jni.h> to implement functions like this:
JNIEXPORT void JNICALL _PACKAGE(bar)(JNIEnv *env, jclass cls, jint i) {
if (bar(i)) {
throwPyFromErrno(env, OSError);
}
}
Next, in Java, you define a public class full of Java wrappers around those C++ wrappers, like this:
public class foo implements InitModule {
public final static native void bar(int i);
}
Finally, in Jython, you can just import the class (which acts like a Python module) from its Java module and use it like any other module:
try:
foo.bar(3)
except OSError as e:
print "Failed:", e
Most of this is standard JNI, but you also have to know things like how to create Jython objects. Ideally, you'll use wrappers for that, so you can just write makePyInteger(env, value) or throwPyFromErrno(env, exctype) instead of doing all the FindClass, GetStaticMethodID, etc. stuff manually.
I don't have any tutorials to recommend. But see jnios for a nice-sized example. The O'Reilly book's Chapter 25. Extending and Embedding Jython seems like it might be a decent primer (although I haven't read it). You'll probably want to read a tutorial on using JNI for Java before trying to tackle Jython.
A different way to solve this problem is to break your single program into two pieces.
The GUI program runs in Jython. When it needs to call the C code, it does that by running a worker program.
The worker program runs in CPython or PyPy, so it can use any of the usual techniques for talking to C libraries: ctypes, cffi, a custom C extension module (maybe using Cython, Boost.Python, SWIG, SIP, …), Weave, etc.
For a simple case, where you just need to call one function, pass it a few strings, and get back a string, it's as trivial as this:
import subprocess
def my_function(*args):
return subprocess.check_output(['python',
'/path/to/worker/script.py'] + args)
(Note that there are a few bugs with subprocess in older versions of Jython, especially on OS X and Windows. If you run into a problem, 2.5.4 and 2.7.0, which are currently in RC and beta stages, respectively, have probably fixed it.)
If you need to make lots of calls one at a time throughout the life of your program, you'll probably want to keep the worker script running in the background, and use some form of RPC to talk to it. This blog post shows how to do it using the bjsonrpc library.
Related
I am having a rather interesting scenario here. Let's say I have a piece of C code stored in a Java String. I need to run this code inside my Java program it self.
Case 1
class Main{
public static void main(String[] args) {
String cCode = "printf(\"Hello World\n\");"
// I need to run the cCode here.
// We are allowed to call a method with params.
}
}
What I think I should do is.
Create a native field in the Main
write the sCode to a file
Exceulte shell commands from Java to compile the c code.
invoke the native method from java
Case 2
I am thinking of doing the above procedure because I know how to do this with JNI if the source code is pre-defined.
class Main{
static {
System.loadLibrary("Main"); // Load native library at runtime
}
private native void sayHello();
public static void main(String[] args) {
new Main().sayHello();
}
}
In the case of pre-written C code. What we do is.
Compile the java class with javac Main.java
Generate the header for C lib. javah -jni Main
Complete the Main.c by writing the C code
Compile the C code with gcc -share -I/path/to/jni -I/path/to/jni_md -o Main.so
Run Main. java Main
Can anyone tell me whether I am taking the correct path (in Case 1) or not? or is there is a better way to do it?
** Note: The key point here is, I am allowed to compile the java code only once. (At the beginning).**
EDIT: After checking the comments and answer from #Dúthomhas I think I should explain one more thing. The reason why I am doing this for a machine learning project. It has been decided that the numeric computation part has a bottleneck and using C as the above-mentioned method is worth the risk of trying it. So security is off the book right now. We just need to do this as an experiment.
Non-answer answer: Don’t do that.
What you are asking to do is a pretty bad idea for several reasons, the two major ones are:
It potentially opens (serious) security holes
Cost to implement likely outweighs the benefit
Asking to embed an entirely different language means adding and linking a library and a lot of code to synchronize the library, plus code to perform static analysis and sandbox the code. In other words, you are asking to implement an entire language on top of what you already have.
It could be argued that C is the base system anyway, upon which the JVM may even be implemented (usually is), but that is not the point. The issue is not the C library, it is the C compiler/interpreter, which is a fairly complex code base as far as simple interpreted programming languages go.
Suggestion: Use Java
The ToolProvider class is specifically designed to provide you with dynamic compilation of Java code. Take a look through it.
Make sure to use the SecurityManager class to properly sandbox the code. (And, if possible, run it in a separate, restricted JVM.)
Suggestion: Use JavaScript/ECMAScript
The ScriptEngine classes are designed exactly for this. Again, Google around for examples. And, again, don’t forget security.
Suggestion: Use an existing library
FScript
Scheme
Tcl
Python
But I really want/must use C
Alas. It is possible to use C, but only with great difficulty. Google around “embedded c interpreter” for small C interpreters that you may be able to integrate into your source. (Good luck, though!)
Let me clarify your two Cases.
Case 1 refers to running a C program out of process. This is not what most people consider "invoke the native method from java". You cannot create a native field in Main. Actually, Java does not have native fields, only native methods. But in Case 1, you don't use native methods, either. You can compile and run any program, written in C or any other language (provided a compiler is available in the runtime environment). To do so, you use shell commands (Runtime.exec()) both to compile the program and to run it. Parameters for the C program can be passed 'on the command line' as specified in Runtime.exec(). It can communicate with your Java process with any IPC of your choice. Typically, we simply use a pipe to read the stdout of the child process.
Case 2 runs the native code in-process, using JNI (Java Native Interface). The flow that you describe here is correct, but you can modify it to use a shared library that is built at runtime. First of all, remove the native method and also loadLibrary() to a separate class. Make sure that the classloader loads this class only after your Main class runs the gcc -o libMain.so command (using the same Runtime.exec() as in Case 1). This way, the static constructor will load the freshly built library.
In both Cases you don't need to recompile your Java. You don't need to run javah to build libMain.so - this is a convenience step only, to guarantee that the C headers are in sync with the Java class. But in your situation, the Java class does not change - therefore the native method signature does not change, too.
Note that Case 1 is easier, especially if you must run different compiled C 'strings', but Case 2 may deliver better performance, if you must call the native method many times.
I am quite new to python and am trying to call python's function from java.
My primary requirements are these:
call should be transparent, in the sense that it should not require modifying .py file simply to enable it to be called from java. I might be given any python file with some functions inside it. I should be able to call any of these functions without requiring to modify .py file.
I want to be able to send arguments of both primitive types (int, String, floats etc.) or non primitive types (HashMap,ArrayList) from java to python function and receive back the returned object (which may of primitive types or non-primitive types) from python to java. I am also using pandas DataFrame and numpy ndarray and hence also want to be able to send and receive corresponding objects to and from java.
I preferably want to stick to CPython instead of Jython because I might need to use newer libraries that might not be available in Jython.
There are several options that I found online. Few are:
Use Jython's PythonInterpreter with which I am able to invoke python function without requiring any changes to .py script file:
py1.py
def square2(list):
squares = []
for i in list:
squares.append(i*i)
return squares
JythonTest.groovy
import org.python.util.PythonInterpreter
import org.python.core.*;
class JythonTest
{
static main(def args)
{
PythonInterpreter pi = new PythonInterpreter()
pi.exec("from py1 import square2")
PyFunction pf = (PyFunction)pi.get("square2")
println pf.__call__(new PyList([1,2,3,4]))[2] //9
}
}
I am very well able to satisfy my needs. But its not CPython.
Use ScriptEngine: This is very similar to PythonInterpreter. But agains its Jython. Also, unlike PythonInterpreter, we cannot work with Jython 2.5+ and cannot directly access PyObjects. So this option can be very well closed.
Use py4j: Cant find example which is as minimal as in case of Jython PythonInterpreter
Use java2python. But not much information is given about calling python from java so that I can conclude whether my above requirements can be satisfied. Can anyone shed more light on this? More specifically if we can write the code as minimal as the one in Jython PythonInterpreter.
Use JPype: However after quick go through I feel I will not be able to write as minimal code as in case of Jython PythonInterpreter. Also I felt the project somewhat not under developement. Is it?
It seems that Jython PythonInterpreter is the best choice if I have understood all above approaches correctly. Have I made mistakes while grasping them? Also is there any other better option?
#Mahesha999, regarding the ability to stick with CPython which seems important now from your last comment:
Jep is a good option to be able to run python code which uses calls to native, like pandas that you mentionned.
You'll need to write some wrapping code because Jep only implements automatic conversion between Java and Python between the most used types, which pandas.DataFrame is not.
However if your use case is not to complex, you can access your pandas objects as numpy.NDArray object by calling DataFrame.values on your dataframe instance, and Jep implement conversion to the Java class it embeds for NDArray.
You can get back to Java values from python code you execute by using Jep.getValue(String pythonVariableName, Class clazz)
For example
Jep jep = new Jep();
jep.eval("import my_script");
jep.eval("df = my_script.function_returning_a_dataframe()");
jep.eval("col = df.a_column.values");
NDArray myCol = jep.getValue("col", NDArray.class);
I do so on a project I coded in Python that I need to integrate as a plugin in a Java application, so far it works.
There is no current answer to this problem. Using CPython relies on the execution of Python bytecodes, which in turn requires that the Python interpreter be embedded in the execution environment. Since no Java runtime comes with an embedded Python interpreter, it really does look as though Jython is the best answer.
Sometimes the answer you want just isn't available!
I want to call Rust code from Java / Android, and I've found 3 variants to do so:
JNI
JNA
JNR FFI
JNI looks good and powerful enough, but you have to
write too much code by hand.
JNA, if not taking into consideration that it crashes on my machine, requires writing Rust struct data type description in Java by hand,
the same problem with JNR FFI.
So I wonder how difficult will be generate JNI code
for traits and struct with macros or a compiler plugin?
This compiler should match traits implementations for concrete struct,
and also struct
#[JNI]
struct Foo {
a: i32,
}
trait Boo {
fn f(&self, b: f64) -> f64;
}
#[JNI]
impl Boo for Foo {
fn f(&self, b: f64) -> f64 {
0f64
}
}
and create Java classes for struct and Java classes with native functions, plus generate pub no_mangle functions that wrap traits functions.
In order to provide #[jni] annotations that work like that you'd need to use a compiler plugin. It would be an awesome tool, but it doesn't exist yet, as far as I know.
There are bits and pieces of tooling lying around that might be helpful, if you want to create a project that does this.
Plugins are currently unstable, and don't work on non-nightly rust; you would probably want to use syntex, which provides a stable interface to compiler plugins. You could also write a raw plugin (see here for the API for those), but most people won't be able to use it.
There's rusty-cheddar, which generates c header files; you could take a look at that to see how it works. The author of that also seems to be working on a more general bindings-generation framework, but I don't know if it's active. You might be able to hook the output of cheddar up to something like JNAerator, but it probably won't create the prettiest interfaces on the java side.
There's also rust-bindgen and corrode, which work in the other direction; they translate c headers and arbitrary c code to rust respectively. I don't know if that's actually helpful.
JNI-sys provides low-level JNI bindings; rust-on-mobile is a small project that uses it. Also see First steps with Rust and Java, a blog post that shows some rudiments of getting things hooked up.
Finally, there's cbox, which lets you work around awkwardness with ownership and FFI.
Finally I created such project (link to github repository) to automate
binding creation.
You can use jnaerator to auto-generate your structure mappings for JNA.
Alternatively, if you want to use JNI (and compile some more native code) you should go with SWIG.
I got an application written in C++ and I am able to extend the applications functionality by writing plugins in C++ for it.
What I basically want to do is to embed Java into this application. This has already been done with Python (not done by me).
I read something about JNI but there is always the speech from a full programm that uses Java classes.
What I'd like to do is, to use classes from C++ in Java to interact with the application.
It's a 3D app in this case, called Cinema 4D.
Is there a way to compile and evaluate Java code while the application is running (in some sort of scripting language) using JNI or anything like it ?
Example imaginary code after the embedding was done:
import c4d.documents.*;
class Main {
public static void main() {
BaseDocument doc = GetActiveDocument();
BaseObject op = doc.GetActiveObject();
if (op != null) {
op.Remove();
}
}
}
This code should interact with Cinema 4D to delete the selected object.
You can embed a JVM within your application. Oracle's official reference book has some more details. The synopsis of it is:
#include <jni.h> /* where everything is defined */
int main() {
JavaVM *jvm; /* denotes a Java VM */
JNIEnv *env; /* pointer to native method interface */
JDK1_1InitArgs vm_args; /* JDK 1.1 VM initialization arguments */
vm_args.version = 0x00010001; /* New in 1.1.2: VM version */
/* Get the default initialization arguments and set the class
* path */
JNI_GetDefaultJavaVMInitArgs(&vm_args);
vm_args.classpath = ...;
/* load and initialize a Java VM, return a JNI interface
* pointer in env */
JNI_CreateJavaVM(&jvm, &env, &vm_args);
/* invoke the Main.test method using the JNI */
jclass cls = env->FindClass("Main");
jmethodID mid = env->GetStaticMethodID(cls, "test", "(I)V");
env->CallStaticVoidMethod(cls, mid, 100);
/* We could have created an Object and called methods on it instead */
/* We are done. */
jvm->DestroyJavaVM();
}
You can do far more sophisticated things if you want (e.g. custom class loaders) but that's about it in terms of the bare minimum needed to get a JVM working within your application.
There seems to be some confusion over whether you want to embed Java into the C++ app or the other way around. I will take each case.
For embedding java into c++ app, you can make a socket call to the java program. On java end, you use SocketServer and on the C++ end, you use the General Socket Layer library. This is by far the easiest and most scalable approach. As your java workload keeps increasing, you keep adding additional jvm. Slightly complicated to deploy but, it works really well.
For embedding C++ app in java. This is simple. You compile C++ app into a shared library and use JNI to invoke it.
What I basically want to do is to embed Java into this application.
This has already been done with Python (not done by me).
The JNI Invocation API supports this, as described by #awoodland. Here's a current link, for Java 6/7.
What I'd like to do is, to use classes from C++ in Java to interact
with the application. It's a 3D app in this case, called Cinema 4D.
For this you could use one of the following:
Java native methods implemented in C
JNA
SWIG
Is there a way to compile and evaluate Java code while the application
is running (in some sort of scripting language) using JNI or anything
like it ?
BeanShell or Groovy, among others, might be of interest to you. Both support dynamically interpreted code that runs on the JVM.
I've been working in something similar lately. What worked for me was using the library jni.h that comes when you install java (Java\jdk[version]\include) and creating a dll with c/c++ code in visual studio. For example:
Test.h
//declare the method you want to implement in c/c++ in the header
//include the jni header
#include <jni.h>
JNIEXPORT void JNICALL Java_Test_print(JNIEnv *, jobject);
//Java_[Class Name]_[Method Name](pointer to JVM, java object);
Test.cpp
extern "C" JNIEXPORT void JNICALL Java_WinampController_printTrackInfo (JNIEnv *env, jobject obj){
printf("Hey, I'm a java method in c (or not?)\n");
}
Then create a dll with Visual Studio and load the dll within a static block.
I didn't try that without compiling the c/c++ code in a dll, maybe there's another way to call the c/c++ code. But that's how you implement it.
Test.java
//declare the same method native inside a class in java
public class Test{
static {
System.loadLibrary("Test"); //load the dll
}
public native void print();
} //after that you just call test.print() normally
So, you just do that and implement java methods with all c/c++ you want.
If you still don't know how to do it, enlighten yourself here:
Java Native Interface Specification - Oracle
Java Native Interface - Wikipedia
For the scenario you are describing JNI is probably the best way to go. You would be exposing the functionality of your C++ app as a DLL that can be incorporated into and used from a Java application.
You probably need to rethink your design. Java is not a good choice for this kind of tasks. There is no eval() function in java standard library similar to eval() from python or shell.
You can create a java VM in C++ code using JNI but it's kind of heavy. There is still issue how to create a bytecode from a java source. You will have to embed a lot of code to compile and run java code in C++. Don't do that. There must be better solution.
You can for example use some RPC (SOAP, XML-RPC, Corba) between your C++ code and separate java code. If you need to do some eval()-like java invocation, you can use Groovy or Jython (both has eval(), access to all standard java library and can run regular java classes).
I have a Python interface of a graph library written in C - igraph (the name of library). My need is to invoke the python modules pertaining to this graph library from Java code. It goes like this, the core of library is in c. This core has been imported into Python and interfaces to the functions embedded in core are available in Python. My project's rest of the code is in Java and hence I would like to call the graph functions by Java as well.
Jython - which lets you invoke python modules with in Java was an option.I went on trying Jython to discover that it will not work in my case as the core code is in C and Jython wont support anything that is imported as a c dll in python code.I also thought of opting for the approach of calling graph routines directly in c. That is without passing through Python code. I am assuming there must be something which lets you call c code from Java, how ever I am not good in C hence I did not go for it.
My last resort seems to execute Python interpreter from command line using Java. But that is a dirty and shameless. Also to deal with the results produced by Python code I will have to write the results in a file and read it back in java. Again dirty way.
Is there something that any one can suggest me? Thanks to every one giving time.
Thanks Igal for answering. I had a look at it. At first glance it appears as if it is simply calling the python script.
Jep jep = new Jep(false, SCRIPT_PATH, cl);
jep.set("query", query);
jep.runScript(SCRIPT_PATH + file);
jep.close();
Isnt it very similar to what we would do if called the python interpreter from command line through a Java code.
Runtime runtime = Runtime.getRuntime();
Process proc = runtime.exec("python test.py");
Concern is how do I use the results generated by Python script. The naive way is to write them to file and read it back in Java. I am searching for a smarter approach.Thanks for suggestion anyway.
Never tried it. But I recently stumbled on a project named Jepp that may be of interest to you.
Jepp embeds CPython in Java. It is safe to use in a heavily threaded environment, it is quite fast and its stability is a main feature and goal.
If you want to call C functions from Java, JNA (Java Native Access) is probably the way to go. JNA allows you to call functions in native libraries without having to write the C glue code (as you would have to when using JNI), and automatically maps between primitive data types in Java and C. A simple example might look like this:
import com.sun.jna.Native;
import com.sun.jna.Library;
public class PrintfWrapper {
public interface CLibrary extends Library {
CLibrary INSTANCE = (CLibrary)Native.loadLibrary("c", CLibrary.class);
void printf(String formatString, Object... args);
}
public static void main(String[] args) {
CLibrary.INSTANCE.printf("Hello, world\n");
}
}
However, things will get complicated with igraph because igraph uses many data structures that cannot be mapped directly into their Java counterparts. There is a project called JNAerator which should be able to generate the JNA source from igraph's header files, but I have never tried it and chances are that the results will still need some manual tweaking.
Also note that a Java interface for igraph is being developed slowly but steadily and it might become useful in a few months or so.
You can use jep.getValue() to retrieve a value from script's global dictionary.
There are caveats to that concerning scope levels in Python, so most people find it clearer to pass a Java class to python and set the return value in that instance in Python. After the script completes, the Java code will then have the result.
For example:
==> Java
class ReturnValueClass {
public int scriptResult;
};
ReturnValueClass value = new ReturnValueClass();
jep.set("retval", value);
==> Python
# do something
pass
# write the return value
retval.scriptResult = some_python_value
==> Java
System.out.println(value.scriptResult);
Hope that helps,
Mike (I wrote Jep)