I have a strange problem. I noticed that on operations where there are variable manipulations in for loops with a lot of iteration, there were big differences in terms of execution time between these two situations:
The program is launched by Eclipse
The program is launched in terminal with a "java -jar ..."
The same loop takes 1 hour with the terminal and 1 minute under Eclipse! (it is about reading a CSV file and putting the columns in a table)
I specify that my program has these VM parameters:
-Djdk.http.auth.tunneling.disabledSchemes="" -Djdk.http.auth.proxying.disabledSchemes="" -Dsun.net.http.allowRestrictedHeaders=true -Djdk.tls.client.protocols="TLSv1,TLSv1.1,TLSv1.2" -Xms2g -Xmx40g -Dhttps.protocols="TLSv1,TLSv1.1,TLSv1.2"
But the problem doesn't come from there I think. To validate this point I proceeded as follows:
I went in the VM arguments of the program launcher in Eclipse and checked that they were there
I looked at the default JVM used by Eclipse in the general settings and I used this path to launch my program in terminal with this command :
"C:\Program Files\Java\jdk-11.0.11\bin\java.exe" -Djdk.http.auth.tunneling.disabledSchemes="" -Djdk.http.auth.proxying.disabledSchemes="" -Dsun.net.http.allowRestrictedHeaders=true -Djdk.tls.client.protocols="TLSv1,TLSv1.1,TLSv1.2" -Xms2g -Xmx40g -Dhttps.protocols="TLSv1,TLSv1.1,TLSv1.2" -jar run.jar
Is there anything that can explain this performance difference? I would like to get it back so that my program can run faster.
Related
I can't get multiple threads to run using JOMP no matter what I try. I actually can't run a JOMP program from the command line no matter what I try either in fact, although ironically it will compile from there and then run in Eclipse! Even in Eclipse though I only have one thread. I've been through the notes from my university course about installation of JOMP carefully, but they have not helped. I'll be more specific though:
Items in quotes below are from those notes:
"There are a couple of websites that tell you how to make jomp run under Eclipse, see http://www.lst.inf.ethz.ch/teaching/lectures/ss10/24/ assignments/assignment_10/eclipse.txt"
This refers to a now broken link. It also seem to be the only link anyone on forums like Stackoverflow refer to when talking about this issue. Apparently it has instructions on runtime settings for Eclipse to allow multiple threads to run, but since the link is currently broken I can't access those valuable instructions.
"All that is required in order to do that is to ensure that jomp1.0b.jar is on the CLASSPATH"
I ran echo %CLASSPATH% at the command prompt to check if it was on the class path and got the following response:
C:\Program Files\Java\jre1.8.0_162\lib\jomp1.0b.jar
On my PC the jomp jar file is in that folder, so it appears I should be able to execute compiled JOMP programs from the command line, but unfortunately that is not the case. By executing one of these commands it should run:
java −Djomp.threads=2 parallel
java −Djomp.threads=2 -cp . parallel
java −Djomp.threads=2 -cp C:\Users\terry\eclipse-workspace\JOMPHello\src parallel
This is the folder the jomp, java and compiled class files are in. I also checked if "parallel" is the fully qualified class name in the way I have set it up in Eclipse, and it does appears to be. So running one of these commands should allow me to run the jomp program from the command line as near as I can tell, but they all return the following error:
Error: Could not find or load main class parallel
Caused by: java.lang.ClassNotFoundException: parallel
(To which I feel like telling Java, "You're not looking hard it enough! It is right in the folder I am running this command from!")
Clearly I am missing something. Can anyone tell me how to get JOMP programs running on the command line, or alternatively knows where there are accessible instructions for how to set up the work around runtime settings in Eclipse?
My implementation of the program seems to run with only one thread, so hopefully that means it is correct, but I can only be sure once I have run it with at least a few more threads.
Thanks,
Terry.
I figured out how to set up the runtime argument in Eclipse. You just have to add the following line into the VM Arguments box in under the Argument tab in Run Configurations for the file:
−Djomp.threads=n
(where n as before is the number of threads you want).
I'd still like to know why it's not working on the Command Line though. It makes me think my Java is set up weirdly.
I am running a batch file which runs a battle in Robocode from within java, and then collecting the results of the battle. The code I am using to run the batch file is:
Process p = Runtime.getRuntime().exec("cmd /c C:\\Users\\Joel\\Documents\\ver5Files\\battle.bat");
p.waitFor();
The contents of my batch file are:
cd "C:\robocode"
java -DNOSECURITY=true -Xmx512M -Dsun.io.useCanonCaches=false -cp libs/robocode.jar robocode.Robocode -nodisplay -battle battles\DT_battle.battle -results resultsv5.txt
exit
This works fine if I run a battle with less than 35 rounds in robocode, however when I try and run one with 35 or more battles the code hangs and never returns to the java code.
I investigated further by running modifying the batch file so that it ran with a display, so I could watch the battles and find out where it was hanging. Once the 35th battle is done, it enters the "Round 35 cleaning up" step, and basically gets hung up there. The program isnt frozen or anything, it just wont initialize the next battle.
I have discussed it with someone who is working on a similar project and they are not having these issues, nor does our code differ significantly for this part. I have tried running it on two different machines, but get the same result on both.
Any advice?
Try to use /k instead of /c and tell me if it works.
Well, if you look at cmd.exe commands you will see it has many options. Some of them are:
/C Run Command and then terminate
/K Run Command and then return to the CMD prompt.
This is useful for testing, to examine variables
You may find more options here.
My question consists of several things I don't understand about the use of "cmd.exe", "/c" when executing sub-processes from Java in Windows. Basically, I couldn't find a good explanation about when and why they're needed.
My specific problems: I have a small framework for sub-processes execution. One use is a Java application which "manages" several other JVMs created by ProcessBuilders. One of the key requirements is that when a sub-process is stuck, or the hosting application is terminating, it must be able to kill the sub-processes.
The problem is, on one hand, doing this:
new ProcessBuilder("java", "...").start();
Causes this:
Could not find or load main class ...
As if the system variables or directory are different (which they're not). On the other hand, wrapping it in a cmd.exe like this:
new ProcessBuilder("cmd.exe", "/c", "java", "...").start();
WORKS, but creates another cmd.exe process, which has a side effect: the child JVM is now a sub-sub-process, and process.destroy(); doesn't kill it (a known bug in the Windows JRE as I found).
This specific problem was handled on a different level, since all those applications are ours and we know their PIDs. But it's an example how cmd.exe makes everything work differently (or prevent the JVM from working at all). So I'd like to know what exactly happens there.
Here the framework itself comes into the picture as well. It's also going to be used by our testing platform. I'd like to provide an API which allows wrapping the command with a cmd.exe /c by a parameter. But, what exactly is the meaning of that parameter? How do the users decide if they want a cmd.exe wrapping?
AND a bonus I'd appreciate: is any of this relevant in other OS? Does it have some kind of an equivalent, say, in Linux?
Came across it again and found the problem, and some additional insights.
#HarryJohnston - good point, according to the javadoc the sub-process inherits the environment from both ProcessBuilder and Runtime.getRuntime.exec(...) APIs so it's not the issue (and java.exe is in fact found so PATH is obviously available).
But it seems certain command line features are lost. Among others it's this kind of %VARIABLE% usage in command line - they're not interpreted unless the command starts with cmd.exe /c.
In this case of class not found (it could also cause NoClassDefFoundError), I had several variables used in the classpath (perhaps I should have posted the entire command, but it's kinda long). When I replaced them with the literary paths, everything worked.
Basically:
java -cp %classpath% Main - bad
cmd.exe /c java -cp %classpath% Main - good
java -cp D:\proj\bin Main - good
So, on the general question of how's the behavior different with cmd.exe:
Variable references like this: %VARIABLE% are only interpreted by cmd.exe.
With cmd.exe you don't really need to split the command into String array of parameters, you can use one long String (array of "cmd.exe", "/c", "the rest...").
The sub-process will be cmd.exe and not the actual executable, so process.destroy() won't kill it. This might be a bug fixed in later versions (I used Java 7 and 8, Windows 7 and Server 2012).
File associations only work with cmd.exe, so you can't "execute" a file that is not a program or script ("CreateProcess error=193, %1 is not a valid Win32 application"). Calling executables defined in PATH does work.
start and call are only available under cmd.exe.
Command level errors behave differently as well: With cmd.exe the error will just be in the output stream, whereas without it the API throws an exception, which might also have a slightly different description.
For example:
nothing will cause java.io.IOException: CreateProcess error=2, The system cannot find the file specified
cmd.exe /c nothing will return output: 'nothing' is not recognized as an internal or external command, operable program or batch file, and a return value of 1 (the only indication something went wrong).
On the deeper question of why, and is this the full list of differences, I still don't really know. Looks like the JVM has its way of running commands, which you can "wrap" with cmd.exe for additional features and protection.
Another interesting fact is when you run a Batch, it's actually run under a cmd.exe (with all its features). From my tests it seems process.destroy() can only kill it if it's not waiting for anything external (if stuck on pause for example), with or without additional cmd.exe /c wrapping, but not if it runs a sub-process - which is consistent with the cmd.exe limitation mentioned in point 3.
I have noticed a very strange problem that sometimes(once per few days) happens to my java program. For some reason, it gets totally stuck. It is multi threaded application, i have got no clue what might cause this.
What are the symptoms:
Even if I have scheduled code to just print something in console, it doesn't show up at all
I run the program by .bat file and I have got shutdown hook, first thing that is done in run() is print in console by Logger. When I click CTRL+C which normally stops the program, completely nothing happens.
I use following settings(-agentlib:jdwp=transport=dt_shmem,server=y,suspend=n,address=5009) to debug program by Intellij IDEA. It was connected before the stuck and after program stucked there is 0 response, Pausing program doesn't give any effect.
I tried to connect to the program with Java VisualVM, after choosing the program, visual VM stucked and there is no response.
I use jre 1.8.0_51, program is run with following arguments:
-Xmx8g -Xms4g -Xmn2g -Xss256k -XX:PermSize=128m -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=15 -Xnoclassgc -XX:+AggressiveOpts
Any clues?
I have a java program that uses ProcessBuilder to call the unix sort command. When I run this code within my IDE (intelliJ) it only takes about a second to sort 500,000 lines. When I package it into an executable jar, and run that from the terminal it takes about 10 seconds. When I run the sort command myself from the terminal, it takes 20 seconds!
Why the vast difference in performance and any way I can get the jar to execute with the same performance? Environment is OSX 10.6.8 and java 1.6.0_26. The bottom of the sort man page says "sort 5.93 November 2004"
The command it is executing is:
sort -t' ' -k5,5f -k4,4f -k1,1n /path/to/imput/file -o /path/to/output/file
Note that when I run sort from the terminal I need to manually escape the tab delimiter and use the argument -t$'\t' instead of the actual tab (which I can pass to ProcessBuilder).
Looking as ps everything seems the same except when run from IDE the sort command has a TTY of ?? instead of ttys000--but from this question I don't think that should make a difference. Perhaps BASH is slowing me down? I am running out of ideas and want to close this 20x performance gap!
I'm going to venture two guesses:
perhaps you are invoking different versions of sort (do a which sort and use the full absolute path to recompare?)
perhaps you are using more complicated locale settings (leading to more complicated character set handling etc.)? Try
export LANG=C
sort -t' ' -k5,5f -k4,4f -k1,1n /input/file -o /output/file
to compare
Have a look at this project: http://code.google.com/p/externalsortinginjava/
Avoid the need of calling external sort entirely.