We Keep Coding

Maven and JVM relation

Though not a direct programming question, it is related to Maven and JVM
While I was going thru this link - http://maven.apache.org/archives/maven-1.x/reference/command-line.html
Update of links: https://wiki.openstack.org/wiki/Documentation/Troubleshooting
https://maven.apache.org/docs/3.0/release-notes.html
it says about MAVEN_OPTS
"Specify additional options using the MAVEN_OPTS environment variable. It is for passing parameters to the Java VM when running Maven. For example, to increase the amount of memory to 1024 Meg for the entire run of Maven, use:
MAVEN_OPTS=-Xmx1024m "
Questions:
How Maven is able to set the JVM properties?
Why does Maven need JVM? It just build the war and JVM is needed only when u deploy the WAR to appserver.right?
Does Maven starts the JVM and stop it after use? IF it stops it after use, what's the point in setting more heap size?
My understanding after research:
(1) How/Why Maven is able to set the JVM properties?
Maven will be able to set the JVM properties using MAVEN_OPTS.
It is to pass the parameter to JVM for that run of Maven
(2) Why does Maven need JVM? Maven is just a build tool..
Maven has to build the WAR file, right? To build the WAR file, you need to compile the code,
and sometimes run the test cases. To compile the code, you need JVM. To run test cases you need JVM.
Don't know where else we need JVM when we do a mvn clean deploy.
[eg of javac passing JVM parameters otherwise: javac -d build/classes -classpath ... -J-Xms256m -J-Xmx1024m java-source-files]
(3) Does Maven starts the JVM and stop it after use? IF it stops it after use, what's the point in setting more heap size?
Yes , Maven starts the JVM and stop after its use. JVM is needed during its run. like if maven build needs lot of memory [like if you do XML processing] then
setting the heap memory using MAVEN_OPTS will be helpful.
Please clarify on my understanding especially on the second question

mvn is a shell script/batch file, and it uses MAVEN_OPTS to do what you'd expect.
Maven is Java. Maven tasks are written in Java.
...? Maven does a lot, and some of those things use memory. It's just a Java program. When it ends the JVM that was running it also ends.
To further clarify point 1, this is what happens at the end of the script:
exec "$JAVACMD" \
$MAVEN_OPTS \
-classpath "${M2_HOME}"/boot/plexus-classworlds-*.jar \
"-Dclassworlds.conf=${M2_HOME}/bin/m2.conf" \
"-Dmaven.home=${M2_HOME}" "-Dmaven.multiModuleProjectDirectory=${MAVEN_PROJECTBASEDIR}" \
${CLASSWORLDS_LAUNCHER} "$#"
Note: why read Maven 1 docs?

Maven is a program written in Java. This program is used to compile files, copy files, make jar files, and a whole lot of other tasks. As any Java program it needs a JVM to run. And as any Java program it uses memory. And as any Java program using memory, it can cause ot of memory errors if you don't start the JVM with enough memory.
mvn is able to set JVM properties because it's a simple shell script that starts a Java program: Maven.
And yes, once mvn exits, the JVM it started to run all the tasks it needs to run exits too.

I am getting the following error when I run mvn on OS X Mavericks: Cannot run program "/bin/sh": error=2, No such file or directory

I am running OS X Mavericks on a MacBook Pro.
I have maven 3.0.5 installed.
When I run mvn, I am getting the following error message.
Error while executing process. Cannot run program "/bin/sh": error-2, No such file or directory
And before you ask:
PATH="/usr/local/apache-maven-3.0.5/bin:/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home/bin:/usr/local/bin:/bin:/usr/bin"
M2_HOME="/usr/local/apache-maven-3.0.5"
M2="/usr/local/apache-maven-3.0.5/bin"
MAVEN_OPTS="-Xms256m -Xmx768m -XX:MaxPermSize=512m"
I am also seeing the following near the beginning of the mvn output:
[WARNING] Some problems were encountered while building the effective settings
[WARNING] Failed to use environment variables for interpolation: Cannot run program "env": error=0, spawn failed #

I had the same problem: exec() failed with ANY executable. Finally remembered that I was using a private jre. That jre was a copy of the original one, just with bin/java set as executable.
I made lib/jspawnhelper executable and it worked!

If you are using eclipse , this is due to the java version. point java to your OS java instead of eclipse java. this will be solved

It seems your system is missing the "sh" shell to start interpreting the script.
I just looked up installed shells on Mac OSX and got this result:
/bin/zsh
/bin/ksh
/bin/tcsh
/bin/bash
/bin/sh
So when "/bin/sh" is missing, try "/bin/bash" for example.
Better:
Check if "/bin/sh" exist
If exist check if the user which runs the command can execute the shell
Because it can be possible that the shell binary exist but isn't executable for you. Then you have to make it executable for the user trying mvn.
Hope that helps ;)

sbt won't assemble Spark

I am having issues assembling Spark using the sbt on my machine.
Attempting the assembly without allocating extra memory either runs out or times out on the garbage collector; the exact issue has been different at different times I have tried. However, any attempt to modify the allocated memory, either through Xmx or Xms, and whether giving more or restricting, fails as the sbt doesn't recognize -Xmx or -Xms as a command.
Here is a sample of the kind of command I have been trying (from the source directory of Spark):
sbt -Xmx2g assembly
Here is the error I have been receiving:
java.util.concurrent.ExecutionException:java.lang.OutOfMemoryError: GC overhead limit exceeded
Use 'last' for the full log.
Not a valid command: Xmx2g
Not a valid project: Xmx2g
Expected ':' (if selecting a configuration)
Not a valid key: Xmx2g
Xmx2g
^
I am running 64-bit Java, version 1.8.0_20.

Try creating a new environment variable SBT_OPTS, with the value "-XX:MaxPermSize=1024m". That should give sbt the memory it needs without producing your error.

Check the docs: Building Spark with Maven.
Setting up Maven’s Memory Usage
You’ll need to configure Maven to use more memory than usual by setting MAVEN_OPTS. We recommend the following settings:
export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m"
Then you can build it with:
mvn -DskipTests package

Maven Surefire 2.13 fails to fork on Windows

I have a project which includes a number of test cases and I compile it using the Maven surefire plugin 2.13. When I set the forkmode = none everything works fine,but when I omit that entry, the default forkmode become once and my project fails:
Forking command line: cmd.exe /X /C ""C:\Program Files\Java\jdk1.6.0_37\jre\bin\java" -jar G:\Gottware-server\core\math\target\surefire\surefirebooter
7758025909061587587.jar G:\Gottware-server\core\math\target\surefire\surefire8735550212668630533tmp G:\Gottware-server\core\math\target\surefire\suref
ire_08386349728753606434tmp"
The system cannot find the path specified.
It looks like it's an OS problem, I am using Windows 7. What can be going wrong? IT's curious that also after succesfull builds I get the same error:
[INFO] -----------------------------------------------
[INFO] BUILD SUCCESS
[INFO] -----------------------------------------------
[INFO] Total time: 15.504s
[INFO] Finished at: Fri Mar 08 15:08:54 CET 2013
[INFO] Final Memory: 13M/353M
[INFO] -----------------------------------------------
The system cannot find the path specified.
In fact I have debugged my last line of my maven of the mvn.bat
cmd /C exit /B %ERROR_CODE%
and this simply does not work anymore returning:
The system cannot find the path specified.
I further went down in debuggining:
U:\>cmd.exe
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
The system cannot find the path specified.

In the line you pasted, I see a doubled double-quote:
Forking command line: cmd.exe /X /C ""C:\Program
^^^^
Maybe you've set a bad value for JAVA_HOME (or some other place where you've defined the Java path). This would also explain why it doesn't fail on other systems: you've got a different (and correct) JAVA_HOME there.

The surefire plugin doesn't always report friendly errors. Couple of things I can think to check:
How much disk space is available in G:\Gottware-server\...? Could you be bumping up against a disk quota (on a NAS share) or is your laptop drive full?
Check where the local artifact repository is located too. Have you hit a quota/run out of room there?
How much memory is available on the build machine? Do you have enough memory available to run the tests in a separate JVM, and if yes, is the JVM configured to have enough memory? You may try using the <argLine> property to configure memory settings for the forked VM. (I've also helped someone who had huge memory settings in <argLine>, more than was available on their machine when running Eclipse, and multiple browser windows, etc. In that case lowering the memory settings for the forked JVM solved the problem.)

I had the same error, and it was there becase I had a bat file launched automatically when cmd launched (and I guess some linked paths issues).
I will quote this answer to explain my solution: https://superuser.com/questions/144347/is-there-windows-equivalent-to-the-bashrc-file-in-linux#answer-916478
My script was called bashrc.bat, which I placed under my %USERPROFILE% folder.
In registry editor (regedit) I located the following entry:
HKEY_LOCAL_MACHINE\Software\Microsoft\Command Processor
And then added a key called Autorun with value %USERPROFILE%\bashrc.bat
When I removed this key from registry, after launching cmd again, the line The system cannot find the path specified. was not displayed anymore and surefire could resume its activities.

Jenkins Server cannot compile because no heap space could reserved

I have a big problem with my jenkins server: I Cant build a maven project, because the java vm cannot start:
Checkout:workspace / /var/lib/jenkins/jobs/SchwarzGoldTool/workspace - hudson.remoting.LocalChannel#b7193fc
Using strategy: Default
Last Built Revision: Revision c2d18fd7a5d7f112163e9440a8e7256a44e32f46 (origin/HEAD, origin/master)
Checkout:workspace / /var/lib/jenkins/jobs/SchwarzGoldTool/workspace - hudson.remoting.LocalChannel#b7193fc
Fetching changes from 1 remote Git repository
Fetching upstream changes from git://.../tsc.git
Seen branch in repository origin/HEAD
Seen branch in repository origin/master
Commencing build of Revision 2b4654302e8222509db5808c9071ec95daf0b495 (origin/HEAD, origin/master)
Checking out Revision 2b4654302e8222509db5808c9071ec95daf0b495 (origin/HEAD, origin/master)
Warning : There are multiple branch changesets here
Parsing POMs
Modules changed, recalculating dependency graph
[SchwarzGoldTool] $ java -Xmx512M -Xms512M -cp /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-agent-1.2.jar:/var/lib/jenkins/tools/Maven_3.0.3/boot/plexus-classworlds-2.4.jar org.jvnet.hudson.maven3.agent.Maven3Main /var/lib/jenkins/tools/Maven_3.0.3 /var/run/jenkins/war/WEB-INF/lib/remoting-2.11.jar /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-1.2.jar 58359
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.
ERROR: Failed to launch Maven. Exit code = 1
Finished: FAILURE
I tried to add -Xmx and -Xms to the VM (as you can see) but that doesnt work either... someone has a idea whats going on there?

The problem is caused by Jenkins failed to reserve enough heap space to kick off a maven build. From what you said, it seems like there are few things which share your VM limit, (I included the estimation of the memory required to run each process)
The VM's OS (~200~300mb)
Jenkins (~min 256mb)
Webcontainers (~256~512mb)
etc. (~100mb)
My memory estimation is rather conservative, but still, it's easily adds up to over 1gb, which leave the available heap for Jenkins to reserve to less than the Xms (512m), hence, failed to kick off a build
Ideally you should increase the softlimit on your VM to a higher value. If that is not feasible, my advice will be to reduce the memory management of the build by change the job configuration in Jenkins to something like this, (Xmx512m, Xms128m) so that Jenkins can kick off a build with only 128m free heap. But this setting may cause out of memory error in the later stage, when the build required a heap which is below Xmx but above the available heap.
Alternatively, you can memory manage other processes I mentioned above or you can setup some configure the virtual memory on your VM.