I'm tasked with converting some Windows scripts to PowerShell versions. The conversion is going good except for some minor issues. Top issue involves a Java call that we use to query a database and generate XML or HTML files. When I run this from a Windows script no problem:
java -Xss1m -cp ../lib/;../lib/jtds-1.3.1.jar;../lib/ojdbc6.jar DBDownloader TMPDBNAME ../sql/MTD_MTTR_Detail_v7.sql ../out/drilldown/%fileName1% >> ..\logs\mtd.log 2>&1
But when I run this from a PowerShell script I get this error:
LASTEXITCODE:1 Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit.
Invalid thread stack size: -Xss1m -cp
The call is made using the & call operator. I wrote a library function for all the script to use and it make a call similar to the following:
*$output = [string] (& $jvApp $jvArg1 $jvArg2 $jvArg3 $dbArg $sqlArg $lastArg >> $logArg 2>&1)
The args for this call are the following:
sqlName: MTD_MTTR_Detail_v7
jvArg1: -Xss1m -cp
jvArg2: ../lib/;../lib/jtds-1.3.1.jar;../lib/ojdbc6.jar
jvArg3: DBDownloader
dbArg: TMPDBNAME
sqlArg: ..\sql\MTD_MTTR_Detail_v7.sql
lastArg: ..\out\drilldown\MTD_MTTR_Detail.xml
logArg: ..\logs\mtd.log
I also set these environment variables:
env:Path: C:\Program Files\Java\jre8\bin;...
env:JAVA_HOME: C:\Program Files\Java\jre8\;
env:CLASSPATH: C:\Program Files\Java\jre8\lib;
What is weird is that the call will work and produce the XML or HTML file if I take out the stack size argument. I've also tried several smaller stack size settings down to 128k with no luck.
While the call works now without the setting, I'm concerned it may fail in the future. I believe they put this due the complex SQL statement used the large amounts of data they were querying and processing.
Any ideas?
SOLVED: When I split up the two args in jvArg1 into two different variables the Java call worked with no stack size error. I guess Java was reading the first arg as one big string and the space in between didn't matter. It must have thought the -cp was part of the stack size argument.
Related
I've been able to create dashboards for small amounts of log data (3mb) with JMeter. However, when trying to create dashboards with large amounts of data (35mb), jmeter will throw a java.lang.OutOfMemoryError: Java Heap Space.
So far I've tried to create an environment variable called JVM_ARGS=-Xms1024m -Xmx10240m but I still do not have enough space.
Is there anything else I can try to create these dashboards? Or is there a way to reduce the number of entries that get written to the log file?
Thank you!
There are 2 possibilities :
Option 1 : your JVM options are not taken into account. Show the first lines or all content of jmeter.log
Option 2 : you have added some dynamic parameter to your http requests that has created a lot of different (name) SampleResult
Edit 8 october 2018:
Root cause was point Option 2
Make sure you've really created the environment variable and it has the anticipated value, double check this by running the following command in the terminal window where you will be launching JMeter from:
echo %JVM_ARGS% for Windows
echo $JVM_ARGS for Linux/Unix/MacOS
You should see your increased JVM heap settings
Make sure to use either jmeter.bat for Windows or jmeter.sh for other operating systems wrapper script
Make sure to use 64-bit version of JRE as 32-bit will not be able to allocate more than 3G heap
Make sure you can execute java command with your 10G heap
java -Xms1024m -Xmx10240m -version
you should see your Java version
Try running ApacheJMeter.jar executable directly:
java -Xms1024m -Xmx10240m -jar ApacheJMeter.jar -g result.jtl -o destination_folder
If nothing helps be aware that you can generate tables/charts using JMeterPluginsCMD Command Line Tool (it is not a part of standard JMeter installation, can be installed using JMeter Plugins Manager)
Is it possible to run an entire, large, powershell script in a java application without calling it externally by launching a powershell process with the -file parameter? (passing it via the encodedcommand parameter won't work either because of the commandline lenght limitation).
I.e. is there a java library that enables you to paste your powershellscript inside your java app and run it?
I currently embed the powershellscript inside the java application and write it to disk, but I'm looking for a fileless approach.
Since you want to pass a large program, piping from the Java program to the PS script is more suitable than using the command line or environment variables. Since I don't know too much Java and don't have it installed, I'll simulate it from the PS command line in the snippet below. Your Java program would read the lines of the program from a list or some other suitable data structure (whereas this example reads it from the file system). I did see hits on google for piping from Java to an externally executed command.
cat my.ps1|
powershell -command {$scr="";foreach ($line in $input){$scr+=$line+"`n"} echo bxb $s cxc}
After your PS stub has received the program rather than echo it, it would execute it: iex $scr.
If you need to pass parameters from Java to PS you could either pipe them along with the program (for example as lines of code that set global variables) or they could be global variables set in stub code. Other, more complex, variations are possible.
I want to use an environment variable as a JVM option when executing java -jar.
The command I want to execute it:
java -XX:onOutOfMemory='echo test' -jar foo.jar
When I run the above command as is, the jar will run.
(If you don't have the foo.jar, you will get an Error: Unable to access jarfile foo.jar error. But this still means that the option gets used correctly).
But when I create an environment variable containing the JVM option, and run the command using that variable.
OOM="-XX:onOutOfMemory='echo test'"
java $OOM -jar foo.jar
Than I get the following error:
Error: Could not find or load main class test'
It seems like the java command is ignoring the quotes around 'echo test'.
After looking for similar questions on SO and on other websites, I tried various variations of using quotes:
OOM="-XX:OnOutOfMemoryError=\"echo test\""
OOM='-XX:OnOutOfMemoryError="echo test"'
OOM=-XX:OnOutOfMemoryError="echo test"
But they all result in the same error.
An article from Oracle concerning JVM options, mentions using a semicolon:
-XX:OnOutOfMemoryError="<cmd args>; <cmd args>"
But the purpose of the semicolon is to separate multiple commands, not command and arguments. So this does not fix my problem.
Does anybody know how I can correctly move the -XX:onOutOfMemory='echo test' option into an environment variable?
When running java, you should quote $OOM
Example:
java "$OOM" -jar foo.jar
See Why does my shell script choke on whitespace or other special characters? on Unix stackexchange for why this is needed.
I have a java program that uses ProcessBuilder to call the unix sort command. When I run this code within my IDE (intelliJ) it only takes about a second to sort 500,000 lines. When I package it into an executable jar, and run that from the terminal it takes about 10 seconds. When I run the sort command myself from the terminal, it takes 20 seconds!
Why the vast difference in performance and any way I can get the jar to execute with the same performance? Environment is OSX 10.6.8 and java 1.6.0_26. The bottom of the sort man page says "sort 5.93 November 2004"
The command it is executing is:
sort -t' ' -k5,5f -k4,4f -k1,1n /path/to/imput/file -o /path/to/output/file
Note that when I run sort from the terminal I need to manually escape the tab delimiter and use the argument -t$'\t' instead of the actual tab (which I can pass to ProcessBuilder).
Looking as ps everything seems the same except when run from IDE the sort command has a TTY of ?? instead of ttys000--but from this question I don't think that should make a difference. Perhaps BASH is slowing me down? I am running out of ideas and want to close this 20x performance gap!
I'm going to venture two guesses:
perhaps you are invoking different versions of sort (do a which sort and use the full absolute path to recompare?)
perhaps you are using more complicated locale settings (leading to more complicated character set handling etc.)? Try
export LANG=C
sort -t' ' -k5,5f -k4,4f -k1,1n /input/file -o /output/file
to compare
Have a look at this project: http://code.google.com/p/externalsortinginjava/
Avoid the need of calling external sort entirely.
I am doing the following in PHP:
exec('java -jar "/opt/flex3/lib/mxmlc.jar" +flexlib "/opt/flex3/frameworks" MyAS3App.as -default-size 360 280 -output MyAS3App.swf');
When I run this from the command line, it runs fine and finishes in a second or two.
When I run this command from PHP exec, the java process takes 100% CPU and never returns.
Any ideas?
I have also tried running the above command with '/usr/bin/java -Djava.awt.headless=true'.
I am running Mac OS X 10.5.5, MAMP 1.7, PHP 5.2.5
Turns out it was a bug specific to the PHP stack MAMP (http://www.mamp.info/).
Turns out any invocation of the JVM following fails under MAMP, e.g.:
exec('java -version');
The fix is to prefix the command with
export DYLD_LIBRARY_PATH="";
Also I realized there's no reason to use that method of invoking mxmlc.
So here's the final, working command:
exec('export DYLD_LIBRARY_PATH=""; mxmlc MyAS3App.as -default-size 360 280 -output MyAS3App.swf');
I manage to get this to work togheter with MAMP. The solution was to include the:
export DYLD_LIBRARY_PATH="";
in the exec call:
$argss = "export DYLD_LIBRARY_PATH=\"\"; /usr/bin/java -jar /Applications/yourjarfile.jar";
$resultXML = exec($argss, $output);
Is there a reason why your using the mxmlc jar file to compile your flex application? have you tried using the executable or an ant task, instead?
Maybe the compiling is taking too long so that your PHP script times out?
Exec is always tricky, on any language :-)
Try to:
use background execution (add &
symbol at the end)
use shell_exec instead
specify the full path to
java executable (may be the one
available to PHP is not the one you
need?)
run a simple HelloWorld java
app to see if the problem is in Java or
in mxmlc specifically
It's strange that java takes 100% CPU. I cannot explain it with any common mistake made when using exec()... try to send it a SIGQUIT to dump the threads, then read the dump -- may be you'll figure something out.