I have a game implemented in Java that was having a problem when running too much code from scripts: depending on the script language, the game could have these "hiccups" where the game would freeze for a couple frames every now and then, making the game "jerky" at times.
After some research, I discovered that was happening when the Garbage Collector decided to run. After some more research and testing, I discovered that using the incremental garbage collector (by using the -Xincgc VM parameter) fixed the problem. Yes, I am aware it makes the GC run about 10% slower, but it is the cost I pay.
With that background information, now what I want to do is package the game in a .jar like I have been doing so far, but I know no way of making the application use the incremental GC without using -Xincgc, and I didn't want to create .bat/.sh files where before the user only had to double-click the .jar.
Is there any way to make the .jar run with the incremental GC without needing some kind of loader (.bat/.sh) or wrapper around it? Is there some way to include that parameter in the .jar?
No, there's no way to do that. VM arguments can only be supplied as command-line parameters to JVM invocation: Java Tool
You can definitely do this with JNLP, which can be configured to add a shortcut to the desktop and automatically keep jars up to date, among other things.
From just a jar file, you can't do this. Adding Java WebStart can allow you to add java VM commands. If you are not against wrapping the jar in an executable Launch4J will wrap the jar in a executable as well. It only makes a windows exe, but it can make it on linux or windows. I haven't found a good alternative for linux.
I don't know of too many java apps outside of internal coorporate or dev stuff that people run via clicking a jar. Even a shortcut can specify this. Fancier options exist that let you create an exe, run as a service, or even wrap the whole thing in an installer... if we are talking windows, similar things exist for linux.
Besides Java Web Start, you could consider wrapping with JSMooth. It gives an EXE which in turn invokes the JVM - you can provide arguments there.
One option - I always wondered - is to re-run the jar program with Runtime.exec() call when the parameters are inadequate? Never tried it but seems possible.
Related
I have an exe that is starting a JVM and loading java classes into memory. I was successfully able to dump these classes with a -javaagent from the running JVM by attaching it to the process and retransforming the classes. Now I am trying to modify the classes as they are loaded. My issue is that my java agent cannot handle premain as I cannot add command line options without modifying the loader executable, which is packed with Themida. By the time my java agent has been attached, the classes to be modified have already been loaded. I was thinking I could use the JVMTI class load hook to capture the class file and transfer it to a separate JVM and modify it with javassist or ASM then return it to to the original JVM via the JVMTI Agent. If I use JVMTI I'd need to load my agent with -agentlib, which brings me back to the problem of having to modify the executable. Can you think of any other ways I could preform this modification? Could I modify the JVM itself to load a java agent always on startup? Is there some other way to force the JVM to always load a java agent? To see if the software was calling any sort of exe in the JRE, I removed all of the exes from j64\bin. The program still loaded without error. When I removed java.dll, the program failed to load entirely, while the program still loads but throws an error when other dlls are removed. If I found which JRE dll was receiving the command line args from the loader, could I implement a dll proxy? Is java.dll the dll I'd want to proxy?
I can't really speak to the idea of proxying the jvm.dll. Sounds like very delicate and error-prone work. Don't know much about Themida either, but from some cursory browsing, you would be deliberately breaking Themida's security intent.
Without the ability to change JVM command line parameters, you don't really have a lot of options. (Rhetorical: What do you do if you need to change heap sizes etc. ?)
However, if you can append and additional execution after the JVM launches, you could use a JVM Attach execution to connect to the running JVM and and load your JVMTI agent via loadAgent. It's a real race-condition threat since your app could very well be under-way by the time you get your instrumentation installed, but as I said, your options are limited.
I would look into how you can use Themida to re-package the application and introduce a proper javaagent command line parameter. Plus, if you don't, you may find that Themida may detect your workarounds and disable them, or disable the JVM altogether.
I know that it is possible to launch java program in Android in a such way:
adb push "hello_world.zip" $REMOTE_PATH/
adb shell mkdir $REMOTE_PATH/dalvik-cache
adb shell dalvikvm -cp "$REMOTE_PATH/hello_world.zip" -Xint:portable -Xdexopt:none -Xverify:none hello_world
It's very good!
But now I want to create Android .apk file. And I want this application to be launched with the same command line parameters "-Xint:portable -Xdexopt:none -Xverify:none" automatically.
I assume that parameters names may differ from this "-Xint:portable -Xdexopt:none -Xverify:none", but functionality must be the same. I have no idea what I have to do and google doesn't want to help me to solve this issue.
Any help would be much appreciated!
I think #Delyan's answer hit the important points, but I wanted to add a few more details that didn't fit in a comment.
The official doc on doing this sort of thing is Controlling the Embedded VM (in source tree). The execution-mode property affects all apps, because the VM is only started once -- as "zygote" -- and all apps are launched by forking that process.
It's possible to change the mode of the VM once it has started, perhaps by introducing a new VMRuntime API call, but I don't think anybody has tried that and I'm not sure offhand how it would interact with the JIT compiler.
Most DEX optimization and verification is performed by dexopt before the app is ever launched, and the results are stored in the .odex file in /data/dalvik-cache, so tweaking -Xdexopt and -Xverify at app launch time may not do much (see dexopt docs; source).
I'm reasonably certain that you can't. You can change the system-wide interpreter mode with setprop dalvik.vm.execution-mode int:portable but you would need root. I don't know of any way to disable dexopt for your process only (that would be a security risk!). You most certainly can't touch the bootclasspath (which is the classpath that your app initially runs under).
I believe you can disable jnicheck for your process but that's about it.
Apps on the device run via app_process and the zygote, you can think of them as restricted wrappers around Dalvik. There are no dalvikvm invocations apart from the initial zygote one - every future process starts with a fork(), so you can't actually change those parameters without messing with the zygote. (I'm simplifying but I'm pretty sure I'm right)
I'm making an application in Java using Eclipse Indigo. When I run it using Eclipse the Task Manager shows javaw.exe is using 50mb of memory. When I export the application as a runnable .jar and execute the .jar the Task Manager shows javaw.exe is using 500mb.
Why is this? How could I fix this?
Edit: I'm using a Windows 7 64 bit, and my system says I have Java 1.7 installed. Apparently the memory problem is caused by a while loop. I'll study what's inside the while loop causing the problem.
Edit: Problem found. At one point in the while loop new BufferedImage instances where created, instead of replacing the same BufferedImage.
Without any additional details about your code, I would suggest using a profiler to analyze the problem. I know YourKit and the one that is available for NetBeans are very good.
Once you run you app from the profiler, you should initially look at the objects and listeners created by your application's packages. If the issue is not there, you can expand your search to other packages until you identify things that are growing out-of-control, and then look at the code that handles those entities.
When you run certain parts of the code multiple times and still see memory utilization after that code stopped running, then you might have a leak and may consider nulling or emptying variables/listeners on exit.
It should be a good starting point, but please report your results back, so we know how it goes. By the way, what operating system are you using and what version of java?
--Luiz
You need to profile your code to get the exact answer, but from my experience when I see similar things I often equate it to garbage collecting. For example, I ran the same job and gave one job 10 gigs and the other 2 gigs..Both ran and completed but the 10gigs one used more memory(and finished faster) while the second(2gig) one, I believe, garbage collected so it still completed but took a bit more time with less memory. I'm a bit new to java so I maybe assuming the garbage collecting but I have seen what you are talking about.
You need to profile your code(check out jconsole, which is included with java, or visualVM)..
That sounds most peculiar.
I can think of two possible explanations:
You looked at the wrong javaw.exe instance. Perhaps you looked at the instance that is running Eclipse ... which is likely to be that big, or bigger.
You have (somehow) managed to configure Java to run with a large heap by default. On Linux you could do this with a wrapper script, a shell function or a shell alias. You can do at least the first of those on Windows.
I don't think it is the JAR file itself. AFAIK, you can't set JVM parameters in a JAR file. It is possible that you've somehow included a different version of something in the JAR file, but that's a bit of a stretch ...
If none of these ideas help, try profiling.
Problem found. At one point in the while loop new BufferedImage instances where created, instead of replacing the same BufferedImage.
Ah yes. BufferedImage uses large amounts of out-of-heap memory and that needs to be managed carefully.
But this doesn't explain why your application used more memory when run from the JAR than when launched from Eclipse ... unless you were telling the application to do different things.
Here's my issue. I have an existing .jar file that I must use in my program. The program, however, is written in Python.
Since my program is taking a long time to run (a named entity tagger on a large development corpus) I profiled it using cProfiler and lined profiled it using line_profiler. It seems that 92% of the time is spent on this task.
I am currently using the following code:
import subprocess as sub
sub.call(["java", "-jar", "-Xmx512m", "MyFile.jar",
featuresFileName, numIterations, featureCutOff])
I read somewhere about subprocess vs Popen and other bits and pieces, but couldn't find a good solution that does not require subprocess or os calls (of course, there may not be any).
I'd really appreciate some advice on the fastest way to run a .jar file from within a Python script. Note, however, that I cannot modify the Java code nor do I have access to speak to the developer of that code.
Alternatively, and I don't know if this will help or if I'm simply grasping at straws here, but perhaps there is a way to keep the process called in sub.call() above in the background, somehow keeping the JVM running (?) so that I can simply invoke the jar file. Maybe that can help reduce startup costs? BTW I am a total Java newbie (mostly C++,C#,Python experience) so my question could make no sense whatsoever - I apologize in advance...
You could try porting your Python to Jython, and then run it all natively in the same JVM (that may or may not work). That way you have effectively zero start up time, and the JVM has enough time to leverage its JIT over time to ideally give you better performance overall.
That indicates that most of the time is spent in this process. It may not be the startup time which is the problem. It may be what it does once it has started.
The only way around this I can think of is to run the process in the background, multiple time concurrently if that is an option. (concurrently rather than running one after another)
Try with "-client" option. It should reduce JVM startup time.
By analysing the manifest file of the jar file you can find out the class name of the jar file which is used. So then you could in principle write your own small java daemon which is listening for new arguments to arrive and calls the main() function of the appropriate class. But it is really worth the effort only if startup costs are the issue.
Is there a way to reconstruct the command line arguments passed to Java within a Java program, including the JVM options and classpath option?
I have a Java program that needs to restart the JVM and manipulate its bootclasspath (i.e. trying to override some system classes). I use the libc system method to invoke the new JVM.
I'm open for better approaches, but Java agents isn't an option.
Why not use a file that has these properties just like the Eclipse ini file and NetBeans conf files. That way you just read these properties and spawn the new Java process with these properties.
Back to your question, this previous answer should do
I agree that futzing with the bootclasspath is generally a poor idea. But...
Grab the code for "java.c" - the C program that compiles down to java.exe. You'll find that it just uses the JNI Invocation API to construct a JVM and call the main method. You could modify and re-compile this to look for particular exit codes, etc. and loop around and re-launch the JVM if required.
Alternatively, Eclipse does this (or at least used to), but having one Java program construct the command line (from a props file, etc.) and launch a sub-process. Again, it hooked the sub-process exit code and used that to decide whether or not to re-launch a new sub-process.
Err... modifying a whole core java class at runtime is a very very bad idea.
Whats wrong with subclassing here? Are you trying to modify an external library, add functionality, or be lazy?