We Keep Coding

How to download all possible maven dependencies so they are local

I work behind a very massive firewall that likes to hiccup on random connections, which makes all work with remote repositories a living nightmare for me!
I am looking to work with a project from Git (this one https://github.com/mrniko/netty-socketio) which heavily utilizes maven for downloading dependencies.
What I would like to do is on another terminal (such as http://cloud9.io or something) download all the maven dependencies so that the entire project can be run standalone.
I have already tried mvn clean install and then zipping up the source folder, but its actually not enough! I still get ClassNotFound related errors when I try to run the project locally in eclipse. And for the record, I did add the compiled *.class files in the build properties, so Eclipse knows where they are. It seems like there are some random classes that get generated dynamically which still aren't present (such as log4j -- and I really don't want to hunt each one down individually)
I am wondering if there is a fully thorough way to download all possible dependencies from maven and then either run a project 100% standalone, or create a local maven server from it?
I am running Java 7 on Eclipse Luna and I do have Maven installed on my windows 7 machine (though again it barely works on remote repositories). I also have a Cloud9 instance which I could use to run Maven commands, then zip up the results for local download.

When you execute mvn clean install, maven downloads all dependencies of currently built project to your local maven repository. This is usually located in
%USERPROFILE%\.m2\repository
When you build your project, maven uses that path, to lookup required dependencies.
If you want do download them all, you can try using mvn dependency:copy-dependencies. Then, you'll find all project dependencies intarget/dependencies directory of your project. This also includes transitive dependencies.
To add them all as eclipse dependencies, you may want to try maven-eclipse-plugin. Using that plugin, you can generate eclipse .project and .classpath files, using mvn eclipse:eclipse command. This will populate eclipse files with required dependencies from maven. You should then import the project to eclipse using Import existing projects into workspace, instead of Import existing maven projects.
maven-eclipse-plugin will add all those jars relative to a folder specified by M2_REPO variable. Just make sure you edit this variable inside eclipse project properties, and you should be set.

I've had to deal with similar issues. I would find that due to changes in firewall policies, occasionally all the .jar files in my project had been updated to be a 1K file that, when opened within notepad++ contained a message from the firewall saying that the download had been blocked.
I recommend looking into Nexus for your local repository management, it means your local projects don't have to go past your firewalls to check for maven updates.
http://www.andrejkoelewijn.com/blog/2010/03/09/getting-started-with-nexus-maven-repository-manager/

Use dependency plugin go-offline task.

Import and use a Maven managed Java library (CitySDK)

I am trying to build a web application based on the project library CitySDK, but i don't know how to import it properly into my own project.
I have created a Maven Web Application and added the CitySDK library as a Dependency, and then Selected the Project, right-clicked and Selected Build with Dependencies. No errors appeared, and in the image below you can see it appeared as a dependency.
However, whenever i try to use some of the classes specific to it, in my own Web Application(called TourismApplication's TestClass.java), the compiler displays an error, that the package is unknown. Could anyone suggest ideas as to what i have missed? I have followed a few Maven tutorials, but without any effects.
The dependencies are as follows:

Ok I did some digging and I think I figured out your problem. Their POM file is incorrect based on the structure of project.
First off, I'll paraphrase what I think the steps you took to get it built in your project were, to ensure I followed the same steps to get it working. These are the steps I took:
Cloned/downloaded the sources from the link you placed in the OP
Built the project into a jar file by running the command they said to use: mvn clean package assembly:single
Installed the artifact in your local Maven repo using mvn install
Added a dependency in your project POM
I tried the same thing you did, using the library in a test source file, to no avail. I looked at the .jar file that was built using their instructions and didn't find any .class files archived into it... it was essentially empty.
Turns out, their src folder structure follows Maven standards, but their POM file indicates the sources are down a different chain of directories. If you examine the build log closely, you see "[INFO] No sources to compile"
The POM.xml file they provide specifies the source directory as
<sourceDirectory>src/citysdk/tourism/client</sourceDirectory>
However, the actual files are at
src\main\java\citysdk\tourism\client
After changing the line in the POM file from the above to (similarly for tests):
<sourceDirectory>src/main/java/citysdk/tourism/client</sourceDirectory>
<testSourceDirectory>src/test/java/citysdk/tourism/client/tests</testSourceDirectory>
rebuilt, and installed, it worked when I tried to use it in my project. Hope this helps.
FYI, I used IntelliJ as my IDE, but it should work the same with Eclipse.

Eclipse makes wrong choice of JAR as project dependency when class presents in more than one library

I work on a big Java application and I'm having problem compiling it under Eclipse Kepler:
the application is based on Maven 3.1.0 and successfully compiles there
I generate Eclipse configuration (project files) using mvn eclipse:eclipse
The problem is that certain classes present in more than one JAR (e.g. imagine a class Http and two versions of this class where newer has more methods) and Eclipse makes a wrong choice of the depending class (older one) for particular project which makes it not compilable.
I tried to move Jars up/down in the project dependencies and it helps in certain situations, however, it means that I'm changing configuration changed by Maven.
Is it possible to configure Eclipse so that is "smarter"? Is it caused by Maven (POM files, a bug in Maven's eclipse plugin, ...) although maven compiles the project without any problems? Any other suggestion?

You should not use mvn eclipse:eclipse. The official way today is to use the m2e plugin (which is much smarter) with File->Import->Existing Maven projects.
When you open a pom.xml file you can see the dependency hierarchy and why a given jar was chosen.
(Note that most but not all standard distributions of the latest Eclipse contains m2e. If your do not you can download it from the marketplace, but it may be easier to download a distribution that has it).

IntelliJ not using external Jar's if not added via Maven when running JSF application?

I have a very simple line of code that is called from an .xhtml file ( in a JSF 2.0 project ) which looks like:
try {
Class.forName("com.mysql.jdbc.Driver");
}catch(Exception e) {
e.printStackTrace();
}
So in my C:\ folder I have the file: mysql-connector-java-5.1.24-bin. I have added this jar as an extenral jar in IntelliJ Idea 12.
When I start my application like this, I get the "Class not found exception." However, when I add this dependency to my pom.xml file:
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.6</version>
</dependency>
I get no exception and class is loaded fine.
Here is a screenshot:
The jar file I underlined with green is the Jar added externally from **File -> Project Structure -> Libraries **
The one I underlined with red is the jar file that is downloaded by maven.
My question is: Why do I get the Class not found exception, although I have added the Jar externally?
In case I require a Jar file that I will not be able to download via maven, what am I supposed to do?
One extra information: I created a sample Java project with one class only, and I added this external Jar file with the method I explained above. No maven, no tomcat, nothing. Just a simple main method. I ran this same code, and it worked fine. So it has to have something to do with IntelliJ and one or more of:
Maven
Tomcat ( Deploying a War file )
Maven and Tomcat?
Edit: When I add the external Jar only and try importing a class from com.mysql.jdbc... IntelliJ IDEA does not come up with any warnings / errors. It finds the class, however after the deployment, it seems like the Class can not be found anymore.

The webapp is deployed and run on tomcat. Tomcat doesn't care about the jars in the IntelliJ source project. What matters to tomcat is the set of jar files under the WEB-INF/lib directory of the deployed webapp. Only those jars are in the classpath of the webapp.
I don't know very well how IntelliJ builds and deploys webapps, especially when Maven is in the picture. But I'd guess that it only deploys the jars that are in the compile and runtime scopes of the Maven project. For an external jar, it's probably possible to configure the IntelliJ project in order to include it in the built and deployed archive.
But anyway, if you use Maven, you must be able to build your war without using IntelliJ. That's the whole point of a build tool: it's used to build a project without an IDE. So all the necessary jars must be known by Maven. So the jar, if not available in the Maven central, should be added to your own local Maven repo (at least), or to your company's private repo.

How do I get maven managed dependencies copied into war\web-inf\lib so I can run my GWT 2.0 app in debug mode within Eclipse?

I am updating an existing project from GWT 1.5.2 to GWT 2.0.0. We use maven 2 to manage our dependencies and do all of our development in Eclipse 3.5.
Because we use maven to manage our dependencies, I do not have all of those jars in the war\web-inf\lib directory as GWT specifies. Instead, they are in the maven repository, just where maven likes them. I have the project set up so that maven can successfully build and launch in either dev or web mode and the application runs correctly.
The problem is that when I launch from Eclipse, I get a java.lang.NoClassDefFoundError. If I manually copy of my dependencies into war\web-inf\lib before launching, everything runs fine, but that doesn't lend itself to a long-term solution. First, if I check all of those jars into our version control, that will subvert much of the value we get from maven. As annoying as maven can be, ditching it is not the answer. Second, having developers manually copy them over every time they want to debug something is ridiculous.
So can I get Eclipse to copy the dependencies into war\web-inf\lib before launching? Is there an alternate solution that I'm missing?
Thanks,
Tony

Running the gwt:eclipse goal will copy the maven dependencies into war/WEB-INF/lib. See the Eclipse Configuration section of the Eclipse IDE Integration documentation of the Maven GWT plugin for more details. Also have a look at this answer about Maven GWT 2.0 and Eclipse.

You should install the m2eclipse plugin and use that to build your project within eclipse. This will invoke maven as an external tool from within eclipse.
Your maven project artifact type should be set to war, which will let maven discover the dependencies and bundle them.
See these links:
force Maven2 to copy dependencies into target/lib
http://maven.apache.org/plugins/maven-war-plugin/
http://maven.apache.org/plugins/maven-war-plugin/examples/rapid-testing-jetty6-plugin.html

a maven aware IDE (idea, eclipse, netbeans) should do this packaging automatically. maybe you:
forgot to enable maven import inside IDE?
did not add these dependencies to the pom.xml (so they aren't included in the 'mvn package' phase)
added wrong scope to dependency declaration (e.g. scope 'provided' or 'tested'), so they are ignored for runtime

If you have the packaging method in your pom.xml set to war it should copy runtime depdencies into target/war/WEB-INF/lib.
Or is your project to build something larger like an ear? If so you should probably split your pom.xml into multi-project format.
As for Eclipse, I'm not terribly familiar with it so I can't really help you there. Are you (or can you) run a maven target (like "install") when you do a run or debug?

This is not supposed to be a problem. If you are using Eclipse+WTP then WTP lets you declare jars listed under 'J2EE module dependency' in the project's properties. Those jars are automatically published to tomcat (see note at the end of this comment)
You need to declare an M2_REPO variable in your eclipse environment (search eclipse help for 'classpath variables'), it should point to your local maven repository, and just add external jars from there to your project and mark them as 'J2EE dependencies'.
I did that with tons of projects, Maven and Eclipse live side-by-side :) (without m2eclipse)
A second option is to try to integrate Eclipse and Maven with m2eclipse plugin, as crowne mentioned, but I found this unnecessary.
There is a third option, if you don't use WTP (but you should). Assuming that you use tomcat, then you can copy those jars to $(tomcat.home)/common/lib. If you can do it just for your private instance of tomcat (on each developer's machine) then you are good to go.
This is supposed to be a low-impact solution that will get those jars into the classpath at runtime, but it doesn't scale well.
note: unfortunately the classpath of Eclipse web projects can be confusing. When you add jars to the regular build path of the project, they are not deployed (copied over) to tomcat. To solve this, WTP defines a special library called Webapp Library - this library contains all the jars in the project's WEB-INF/lib folder, as well as all the external jars that are marked as J2EE Module Dependencies. All those jars find their way appropriately to WEB-INF/lib when published to tomcat.

Quite simple:
1. Create a "lib" folder under your project like this:
"\src\main\webapp\WEB-INF\lib"
2. Copy needed "jars" etc that you want included inside your WAR bundle folder.
3. Invoke your maven build as you normally do. I use "mvn install", which creates builds the war file.
If you examine the WAR file, your jars that you included in step-1 and step-2 will be there.
Cheers.