I have a Java program which depends on a native library that crashes randomly (and therefore bring down the whole JVM).
This library is about I/O, therefore can be easily isolated from the rest of the program with a second JVM.
I want to split the program in two so that the main program keeps running, but the native library can be restarted after crash. The question is: what is the best way to let these 2 JVM communicate? One of them will crash and be restarted but this should not affect the other JVM (besides some timeout with the I/O because of the crash and restart).
Is RMI crash resistant? Should I use a socket? A memory-mapped file?
I would use a persistent JMS Queue between the processes. This will be the most crash resistent. (Fixing the library is the best option)
Related
I have a Java 1.6 application that accesses a third party native module, through a JNI class provided as the interface. Recently we noticed that a SEGFAULT is occurring in the native module, and is causing our application to crash. Is it possible to catch and handle this event, at least to log it properly before dieing?
I tried both Java techniques in the article from kjp's answer. Neither worked. Attempting to install a signal handler on 'SEGV' results in the exception
Signal already used by VM: SEGV
The shutdown handler I installed simply failed to fire, presumably because of what the IBM article states:
Shutdown hooks will not be run if
Runtime.halt() method is called to terminate the JVM. Runtime.halt() is provided to allow a quick shutdown of the JVM.
The -Xrs JVM option is specified.
The JVM exits abnormally, such as an exception condition or forced abort generated by the JVM software.
If all you want to do is log and notify you can write a script which runs your application. When the application dies, the script can detect whether the application terminated normally and from the hs_errXXXX file which has all the crash/SEGV information and mail it to someone (and restart the application if you want)
What you need to do is to run the faulty JNI code in another JVM and communicate with that JVM using RMI or JMS or Sockets. This way when the library dies, it won't bring down your main application and you can restart it.
Based on several weeks of research at the time, as well as conversations with JVM engineers at a conference this is not possible. The system will not let you install a SignalHandler on the SEGV signal.
I have coded a Java application that through JNI calls a linear programming solver written in C++. After a number of successful calls I get an EXCEPTION_ACCESS_VIOLATION error and the application terminates.
How can I dismiss the error and keep the Java application operating?
First of all, if a dll crashes then the JVM will terminate.
This is one of the drawbacks of using JNI in the first place. So it is generally not as simple as swallowing some exception from the dll.
In your case the error hints towards unitialized memory. You should look towards out of bound indexing in your code.
You should post your code though in order to get more help
There is no way to keep JVM running if there is access violation happened in native code.
There are several things you can do.
Best - carefully debug your lib without JNI, with small standalone C++ client, and find the reason of this violation, and add excessive checks around that (incorrect pointer usage, incorrect memory cleanup etc)
Run your DLL in a separate JVM process, which is purely responsible for providing this solver functionality, communicate with second JVM using sockets, tcp-ip, soap, rest - whatever you prefer. Than, use some like Java Service Wrapper to run this second JVM and configure it in the way that it restarts the JVM if it crashed.
I wrote a wrapper application in c# NET that runs when the .jar file is running, closes when the .jar file closes, etc. This was basically to allow for our web panel to be able to query the executable to find out if it was actually running or not.
I have seen some other panels specifically intended for this software that have an option to reduce the memory usage of it when no one is connected. The java application (Minecraft) basically scales the RAM usage based on the size of the player world rather than how many players are connected. When no one is connected, it should be perfectly fine to reduce the usage.
So is there any way to reduce the RAM usage programatically from C# NET for a Java application?
AFAIK, there is no way to tell a JVM to give regular heap memory back to the operating system ... apart from telling it exit completely.
No.
Why not? Because you can't control the Java-Program in that way for two reasons:
You can't control what the JRE does with it's memory and how the GC is working.
If minecraft.jar requests 512MiB of RAM, he gets 512MiB of RAM. You can't just go all Hey, there's no one connected so I disallow you to allocate memory on an application. I mean, you could...but I don't think you want that (that would trigger exceptions and odd side-effects).
Edit: The only rather easy way to achieve this behavior would be to change the program. Since Minecraft is not free/open-source software, the only thing you could do is file a bug/feature request. Maybe even with extended information and a layout concept on how to achieve better memory usage.
I mean, I'm pretty sure that this could also be achieved with heavy usage of reflection via a Java program...but things go pretty fast downhill from there.
I assume the latest update version of java would provide better performance.
I am looking for a way to implement isolation of software components from endless loops or memory leaks. Android isolates each app in it's own process, Google Chrome isolates each tab in it's own process.
My primary drawback is that java takes so long to start and also I would like to reduce memory consumption.
Is there any alternate build or more controlled startup that will accomplish this?
If quick startup is your goal, Java on a PC may not be your best bet. It's going to take a few seconds because that's how long it takes to load the VM from disk.
If you want your app to start more quickly it's easy to get a splash screen up, just create a module that only loads your splash screen, waits for it to fully display then uses reflection to link to your "Real" main module.
(Use reflection because otherwise it will pull in your entire program through references before it starts the main one--at least that's how it used to work).
If you're talking about run-time performance, you won't get quicker by changing languages, Java's about as fast as you can get. You MIGHT be able to get a boost by converting to C/C++ and rewriting it to suit those platforms (Less OO, stack allocations instead of heap, etc), but otherwise none of the other languages in general usage are close to Java in speed.
If you really need the quick startup, depending on what you are doing there may be some tricks. I've seen projects that try to keep a Java VM running in your toolbar and allow you to make requests (tell it to start an app). This was faster but made additional requirements of the user (Loading this additional tool)
Another possibility--if you are constantly starting up/shutting down small tasks and that's the reason the startup bothers you then you can definitely speed it up by keeping it running invisibly. Just have your Java app open a socket and listen for commands then create a little .EXE or shell script that can start your program if it's not running or send commands to that socket if it is. This would completely eliminate startups after the first run.
In general, Java has a much longer startup time than other languages. If you are sticking with Java on a desktop app, a lot of stuff like startup time is determined by the JRE installed on the client's computer, which you can't control.
As to "endless memory leaks"... Java doesn't leak memory. If your program does, fix it.
This is a second answer because it's completely different and my other got too long :)
Try compiling it--I think GCC can compile it. This could almost completely eliminate your startup. I believe Jikes used to be a windows java compiler by IBM, but I don't know if it's still maintained.
Note that compiled code will probably run slower than JVM code for long-running apps.
We have a curious problem with our java processes dying.
The application doesn't stacktrace, or write anything to the logs, the process just randomly dies. It's a heavily used application, but the problem only appears about once a month.
We're currently looking into using Process Monitor but any other suggestions would be welcome.
Edit:
It's a distributed Java application, running on Weblogic with an in-house web framework (Yes, this is a terrible idea, but it's been running for eight years), connecting to Oracle.
-
Out of Memory?
Our logs would catch java.lang.OutOfMemoryException, according to Brian Agnew.
Write crashes to a log? I don't think Java ever gets the chance, the death is happening at a process level, rather than Java exiting.
Can you wrap it in some shell script that captures the log files (stdout/stderr) and the exit code (which should give some indication as to how it died) ? On JVM exit you can also capture machine level stats using WMI
IF the VM itself is crashing it'll leave behind an hs_err_pid... file that contains stacktraces, machine-level debug info. You can then use that to diagnose the VM issue. See this blog entry for further information.
If the problem is related to the app's behaviour, it may be worth looking at JConsole, although from your description of the issue, this sounds much more like a low level VM issue.
(I assume you're on the latest VM for your Java version number etc.)
You can use a Linux NAGIOS Server to monitor the health of your Windows machines and services! Have a look at: nagios-monitoring-windows.
If you have such problems with your java app! You should test it and debug it! Applications shouldn't die without a trace! Look for logfiles! From which vendor is the app? Or is it self written? Try to enforce another Log4J/Logger/Debug Level. Monitor your System with cacti etc. to reduce the possibilities for such a crash. Talk to the software vendor.
Is enogh memory available? Maybe the app runs out of memory? Is it a standalone java process or a java process from a tomcat/jboss server?
Have you written down the crash times to a log? Appear they in different time-slices? Or appear they nearly time-circular?
VisualVM is a new tool which makes monitoring Java applications easier:
https://visualvm.dev.java.net/description.html
"VisualVM is a tool that provides detailed information about Java applications while they are running. It provides an intuitive graphical user interface that allows you to easily see information about multiple Java applications."