I have a large body of C++ code that I've wrapped with SWIG and am calling it from Java. The C++ code makes liberal use of boost smart pointers.
Some of my JUnit tests complete but then experience seg faults during cleanup. The stack trace indicates a memory error in an object's finalization, but it's happening in the JNI code generated by SWIG and seems to be associated with the smart pointer reference counting.
I would like to be able to step through all layers of the code. Is this possible? I would also be very happy to hear others' experiences with this sort of problem.
You can attach a second, native-code debugger to the executing Java code. For example, with Visual Studio, you can attach via "Debug>Attach to process."
SWIG supports smart pointers, but you have to manage their lifetime explicitly on the Java side. If you obtain ownership of a smart-pointer object on the Java side, you must delete it. Bear in mind that Java has no object temporaries that go out of scope, so you won't be able to use expressions like f().g().h(), in which the return values at each level are smart pointers.
As a failsafe, the SWIG-generated finalizer will attempt to delete it for you if garbage collection occurs. That failsafe will be harmful if the object has already been deleted on the C++ side.
Related
I have been working in java for a while now, and want to learn how c++ works when it comes to compilation and executed.
I was wondering if there is a way to convert compiled c++ class into .class files in java and vice versa. I am interested in a single format that can be used both by java code as well as c++ code that can be directly executed to see the results.
Almost everything can be done if you are ambitious and stubborn enough, however, the question lies usually in time and cost and features..
In its core, Java language is a subset of C++. There are some syntactic sugars added that may make you feel that there is "something more", like anonymous class implementation or hidden pointers to outer classes, but it is just a thin layer of syntax, which is irrelevant once the code gets compiled.
After compilation, C++ code is represented by machine code. Java's bytecode of course is translatable to machine code - simply by the fact that JVM executes it and that the jitter can recompile it on the fly into machine code..
So, roughly speaking, every Java code, compiled or not, is translatable to C++.
However, there are some code constructs in C++ that can be compiled into machine code, but that maybe are representable in Java's bytecode, but that cannot be represented back in the Java language. There are lots of it: from some easy to go around like passing parameters as references, to more complex ones like pointer (TheList->) arithmetics, to some really painful to translate like multiple inheritance, custom memory management (that is, overloaded operators new and delete), or some wicked types like unions.
So, clearly, C++ code is not translatable to Java. Clearly, C++ compiled code is even more not translatable, as C++ compilers often optimize the products thorougly, so that it is very hard to guess what was the 'classes' or 'functions' like..
However, if you limit the C++ language, and restrict yourself to not use any of those hard-to-translate constructs (see 'TheList'), then you can make the C++ code translatable. Again, code, but not binaries.
This is not all though. The 'translatability' is one thing, but the other is: will it run? The most distinctive runtime difference is the GarbageCollector. Let's say you actually managed to translate some Java code into C++, and you lined it up with C++ application. Your Java/C++ code executes and creates some objects. Who will clean them up? Typically, there's no GC in C++. Your Java code will therefore leak -- or you will have to provide/implement some kind of GC for the Java/C++ code.. Not pretty. Of course you can limit Java code to not create any objects, d'oh.
Do not get me wrong: even those hard-to-translate things like pointer arithmetics etc are translatable: you can generate tons of helper/wireup code that will replace them with 'proper things of the second platform'. It will, however, be ridiculously complex and slow. I don't think anyone sane will ever try.
So, the only thing that would seem to be left available is very-limited-C++-code <-> somewhat-limited-Java-code. If we cut down the question to this, then yes, that should be translatable. But..
What does it mean to translate code? How'd you do it? You have to read, process, analyze the source code, and then somehow produce the other code in the other language. Well, ok. Producing code is simple, it's just text. But, have you ever tried to analyze code? Long story short, let me just tell you that reading/parsing Java code is at least an order of magnitude easier that reading/parsing C++ code. Java was partly desined to be easily parsable by relatively simple algorithms. If you drop any attempts to optimize, writing a Java parser/compiler is relatively simple thing. On the other hand, C++ was not. Like Java, to parse C++ properly you'd have to effectively create a custom C++ compiler, but also a preprocessor. To some extent, you might also need to implement some parts of the linker. To make thins a little worse, C++ evolved from older languages and is literally packed with some once-in-your-lifetime-used features that make the syntax really difficult to accurately process (ie. have you ever used alternate token set? ..and this is only beginning:)). Do not get scared too much, though! I just want to give you a feeling what you try to touch. You probably would'nt need to write it. Such already tools exists, both for Java (really many, actually) and for C++ (few, and I bet the reasonable ones are not-fully-for-free.. or maybe you could use the GCC toolset probably..). They produce machine-processable representations of the sourcecode, and if you really want to do some translating job, I'd suggest you start there.
Of course my knowledge can be off by a few years, and maybe someone already has written some moreorless working translator - I'd love to see it!
If not, I think it is not worth it. Try embedding Java's runtime in your C++ app, or talk from Java to C++ DLLs via JNI. It is much simplier!
Compiled C++ code is loaded by the OS. That is, the C++ linker generate OS dependent executable modules. Whereas Java .class is loaded by the JVM. JVM executes Java byte code.
If you want to make a Java byte code loader/runner, you could start from JVM source code.
=> link
If you are aimed to load/execute compiled C++ code in Java envirionment, following are required.
(Assuming 32bit Windows platform)
. PE parser/loader => link
. X86 CPU instruction parser(?) => link
. X86 instruction to Java byte code translator
In short, its almost impossible.
For simple C/C++ vs. Java interoperations, JNI will do. link
I think, JNI will be useful:
"The Java Native Interface (JNI) is a programming framework that enables Java code running in a Java Virtual Machine (JVM) to call, and to be called by, native applications (programs specific to a hardware and operating system platform) and libraries written in other languages such as C, C++ and assembly."
Here are good tips how to program a simple example of using the Java Native Interfac (write a Java application that calls a C function).
Converting already compiled C++ code to JVM bytecode is not an easy task, especially if you want to be able to use compiled C++ code from multiple platforms.
It will probably be easier to make a JVM-backend for clang instead. The drawback here is that people who wants this functionality must use your compiler.
In short, if you want to write code targeting the Java virtual machine, then use a language and compiler already made to do it. Like Java...
I'm new to Java, and was told to use the Java Native Interface to run some code I wrote in C.
Now, this might be a stupid question, but what's the point of the JNI ? Can't I simply execute my process from a Java UI program and get its stdout to parse ?
Also, I've read that the use of JNI might cause security issues. Do these issues directly depend on the quality of the invoked code ? Or is this something deeper ?
Thanks.
what's the point of the JNI ?
It enables you to mix C and Java code within the same process.
Can't I simply execute my process from a Java UI program and get its stdout to parse ?
A lot of things that can be achieved by using JNI can also be achieved by using inter-process communication (IPC). However, you'd have to ship all the input data to the other process, and then ship all the results back. This can be pretty expensive, which makes IPC impractical for many situations where JNI can be used (e.g. wrapping existing C libraries).
Also, I've read that the use of JNI might cause security issues. Do these issues directly depend on the quality of the invoked code ? Or is this something deeper ?
The point here is that the JVM does a lot of work to ensure that whatever Java code is thrown at it, things like buffer overruns, stack smashing attacks etc can't occur. For example, it performs bounds checking on all array accesses (which C doesn't).
On the other hand, JNI code is a black box to the JVM. If there's a problem with the C code (e.g. a buffer overrun), all bets are off.
Can't I simply execute my process from a Java UI program and get its stdout to parse ?
Do you think it's always appropriate to start a new process every time you want to execute any native code? Do you really want to be transferring potentially large amounts of data between processes? (Imagine a native image transformation.)
Also, I've read that the use of JNI might cause security issues. Do these issues directly depend on the quality of the invoked code?
Yes. Basically native code has less security sandboxing than Java running in a JVM. If the code has security bugs (e.g. buffer overflows) then clearly that will affect the security of your overall app.
I should say that it's relatively rare for Java developers to need to worry about JNI - I've certainly only touched it a couple of times in my career. You may also want to look at SWIG if the need arises.
Can't I simply execute my process from a Java UI program and get its stdout to parse ?
That would depend on what you are calling.
Note that you cannot just call programs via JNI, but library code.
In addition to that, spawning new processes is relatively expensive and managing multiple processes is complicated.
I'm writing a wrapper in Java for a C++ program. The wrapper is done using SWIG. I have the following problem: when I call a function from Java which create a big object in C++, Java doesn't "see" that it has allocated a lot of memory since it's not allocated in Java's heap. The problem is that the garbage collector is not called when the object is deleted as from Java side there is plenty of free memory. What I have tried is to implement what is described here: http://www.swig.org/Doc1.3/Java.html#java_heap_allocations. The idea is to allocate memory space for C++ in Java's heap. As I'm not interested to use that for every new, I have renamed the new and delete and use them explicitly from my C++ code where needed.
This mechanism seems to work (I can see in that Java's heap is growing and shrinked by the garbage collector) but unfortunately I have a random crash whis seems to occur during a memcpy.
If I invoke the garbage collector manually my program is working but it's not a very clean method.
Thanks for any clue.
In fact I didn't notice the following line in swig website:
If you are going to use optimisations turned on with gcc (for example -O2), ensure you also compile with -fno-strict-aliasing
This seems to have solved the problem
Couldn't you simply manually invoke the C++ object destructor via SWIG? This seems to work for me.
I develop applications/programs in C/C++. I am more versed in these two languages and love being a C++ developer. I am wondering how to create a Java program that contains all my C++ code.
I mean, I would like to wrap all my C++ code (that is already developed) inside Java class. But clueless how to do it.
Please post your responses or methods/steps on integrating C++ inside Java.
(using JNI is the way, but I could not figure it out on www how to use it)
FYI, I use Eclipse IDE to develop.
How and what packages should I include in my project workspace?
Instead of JNI, or JNI with some assist from an automatic wrapper generator like SWIG, or even JNA, you might consider separating the C/C++ and Java into separate processes and using some form of IPC and/or Java's Process abstraction to call to a program written in C/C++. This approach abandons "wrapping," so in some sense it isn't an answer to this question, but please read on before down-voting. I believe that this is a reasonable answer to the broader issue in some cases.
The reason for such an approach is that when you call C/C++ directly from Java, the JVM is put at risk of any error in the native code. The risk depends somewhat on how much of the native code is yours and how much you link to third party code (and how much access you have to the source code of such third party code).
I've run into a situation where I had to call a C/C++ library from Java and the C/C++ library had bugs that caused the JVM to crash. I didn't have the third party source code, so I couldn't fix the bug(s) in the native code. The eventual solution was to call a separate C/C++ program, linked to the third party library. The Java application then made calls to many ephemeral native processes whenever it needed to call the C/C++ stuff.
If the native code has a problem, you might be able to recover/retry in Java. If the native code is wrapped and called from the JVM process, it could take down the entire JVM.
This approach has performance/resource consumption implications and may not be a good fit for your application, but it is worth considering in certain situations.
Having a separate application that exercises the functionality of the C/C++ code is potentially useful as a stand-alone utility and for testing. And having some clean command-line or IPC interface could ease future integrations with other languages.
As another alternative, you could get into native signal handling to mitigate the risks to the integrity of the JVM process if you like and stick with a wrapping solution.
If you want to call C++ from Java, you'll need to use JNI - Java Native Interface.
Be warned that you lose some of the benefits of the garbage collector, since it can't deal with your C++ objects, and your code won't be portable anymore.
Maybe you'd be better served by learning to write 100% Java and leaving C++ behind, but that's just a suggestion.
You can't "just wrap it", you have to write some C/C++ glue.
For starters, SWIG can do most of the works for you.
There are plenty of tutorials for doing exactly what you want to do. For example, check out: http://www.javamex.com/tutorials/jni/getting_started.shtml
There are also plenty of caveats of using JNI. I've recently started working with it (just for fun, really), and it tends to be a lot less fun than I had first anticipated.
First of all, you have to deal with cryptic code such as:
#include "test_Test.h"
JNIEXPORT jint JNICALL Java_test_Test_getDoubled(JNIEnv *env, jclass clz, jint n) {
return n * 2;
}
Second of all, it tends to downplay one of the primary reasons why you use Java in the first place: WORA (Write Once, Run Anywhere). As duffymo mentioned, there can also be issues with the garbage collector, but I think that in recent years, the JVM has gotten pretty smart about JNI integration.
With that said, to port all of your C++ code to JNI, you'd need to refactor your interfaces (and maybe even do some internal gymnastics). It's not impossible, but it's really not recommended. The ideal solution is just re-writing your code in Java.
With that said, you could also "convert" your code from C/C++ into Java programatically, and there are multitudes of such utilities. But, of course, machines are dumber than people and they are also bound to make mistakes, depending how complex your class is.
I would avoid JNI because it's tedious to write, verbose, and just an altogether pain. Instead I'd use JNA library which makes writing native integration so simple.
https://github.com/twall/jna/
Good luck.
You can write C++ code through JNI but there isn't a direct mapping from C++ classes to Java classes.
I've used JNI to fix problems found in the android SDK (specifically, an incredibly slow FloatBuffer.put implementation) and I may end up using it for some performance critical areas. My advice would be to be use it sparingly and in a duck in, do the performance critical stuff and leave, without doing any memory allocation if you can help it. Also, don't forget to measure your code to see if it really is faster.
Out of interest, what platform are you developing for? The only platform where it would make sense to wrap a lot of C++ code in a light java layer would be Android - on other platforms, just compile in C++ and have done with it.
JNI module is not a Java classes. It's C. Using JNI incur many restrictions and some Java environment doesn't support JNI well.
There is no supported way of "wrap my C++ code inside Java class" (EDIT: I mean, without JNI noway but JNI is problematic.)
You could investigate custom C++ compiler emit Java byte codes, but nobody (include me) will recommend this approach.
BridJ was designed on purpose for that (and it's supported by JNAerator, which will parse your C/C++ headers and spit out the Java bindings for you).
It is a recent alternative to JNA, with support for C++.
I would like to know if theres a possibility to call C++ methods from java.
Pretty much I want to be able to read memory processes from java.
I know c++, but I Would like to use a higher lvl like java, but still be able to hack into processes memory.
Any clues?
[]'s
Here's the JNA project, which lets you call any function exposed in a DLL, without writing any JNI code.