We Keep Coding

How to download all possible maven dependencies so they are local

I work behind a very massive firewall that likes to hiccup on random connections, which makes all work with remote repositories a living nightmare for me!
I am looking to work with a project from Git (this one https://github.com/mrniko/netty-socketio) which heavily utilizes maven for downloading dependencies.
What I would like to do is on another terminal (such as http://cloud9.io or something) download all the maven dependencies so that the entire project can be run standalone.
I have already tried mvn clean install and then zipping up the source folder, but its actually not enough! I still get ClassNotFound related errors when I try to run the project locally in eclipse. And for the record, I did add the compiled *.class files in the build properties, so Eclipse knows where they are. It seems like there are some random classes that get generated dynamically which still aren't present (such as log4j -- and I really don't want to hunt each one down individually)
I am wondering if there is a fully thorough way to download all possible dependencies from maven and then either run a project 100% standalone, or create a local maven server from it?
I am running Java 7 on Eclipse Luna and I do have Maven installed on my windows 7 machine (though again it barely works on remote repositories). I also have a Cloud9 instance which I could use to run Maven commands, then zip up the results for local download.

When you execute mvn clean install, maven downloads all dependencies of currently built project to your local maven repository. This is usually located in
%USERPROFILE%\.m2\repository
When you build your project, maven uses that path, to lookup required dependencies.
If you want do download them all, you can try using mvn dependency:copy-dependencies. Then, you'll find all project dependencies intarget/dependencies directory of your project. This also includes transitive dependencies.
To add them all as eclipse dependencies, you may want to try maven-eclipse-plugin. Using that plugin, you can generate eclipse .project and .classpath files, using mvn eclipse:eclipse command. This will populate eclipse files with required dependencies from maven. You should then import the project to eclipse using Import existing projects into workspace, instead of Import existing maven projects.
maven-eclipse-plugin will add all those jars relative to a folder specified by M2_REPO variable. Just make sure you edit this variable inside eclipse project properties, and you should be set.

I've had to deal with similar issues. I would find that due to changes in firewall policies, occasionally all the .jar files in my project had been updated to be a 1K file that, when opened within notepad++ contained a message from the firewall saying that the download had been blocked.
I recommend looking into Nexus for your local repository management, it means your local projects don't have to go past your firewalls to check for maven updates.
http://www.andrejkoelewijn.com/blog/2010/03/09/getting-started-with-nexus-maven-repository-manager/

Use dependency plugin go-offline task.

How to include library jar in git project

We have a core set of java libraries, published as jar files, that are used in multiple client projects. Both the library projects and the client projects are on our git server (GitLab EE). Sometimes clients and third-parties (such as sub-contractors) have access to the project source code, but we don't want them to have access to the source for the core java libraries.
So what I need to know is how to set up some kind of dependency or similar in git so that we can automatically pull in the latest version of the library file? I've been looking at git submodule, but that looks like it will pull in, or at least give access to, the full source of the library projects. I've also started looking at Maven, but I'm unclear whether that also needs access to the whole repository. Is there another/better way to do it? Or will Maven actually do what I want? Or do I need to publish my jar files somewhere else, and if so how can I automate that process, as I don't want to have to manually copy the jar file somewhere every time it's updated (which is often)?

Maven and a binary repository like Nexus is the common way to do what you're asking. Nexus stores binary files like .jar's .dll's etc and Maven has the capability to publish and retrieve specific versions of these binaries and include them when building the project. As Ryan commented, Maven Central is a public binary repository that you could publish to if you wanted the whole world access to your .jars, but with Nexus you have fine grained access control over your binaries.
If you build your own projects with Maven, you can configure it to build, test, and publish right to your Nexus repository with a single command mvn deploy. Then clients just have to update their project to point to the new version and Maven will download it and use it on the classpath. It's all very seamless, and I would definitely recommend refactoring your build process to use Maven or a similar product.

maven project checkin in svn with local jar

I am working in a application which consume a webservice. I generated the webservice client using axis2codegen. Since it is generated code there are lot of duplicate codes exist. As per suggestion on code review we have to remove it and add it as jar. I created a jar and add to the local repository of the maven. But the problem is how can i check in the locally generateed jar to the svn, so that the user who check the project does not need to add it to the local repository manually as i did it.
Thanks in advance

Storing the .jar files in subversion isn't really the "maven way". You might want to consider setting up a corporate maven repository using something like nexus or artifactory and then deploy your .jar artifacts there instead.
Once that's done you can distribute a corporate settings.xml file pointing to this repository (or include it in each projects pom.xml file) and you can then manage your generated code like any other maven dependency.
HTH

Don't commit jar into SVN: dependency management has introduced the concept of "artifact repository" separating binaries from sources:
binary are saved into artifact repository
source are saved into source code management repository
In order to share the jar with your pears you should:
install the jar in a corporate maven repository like artifactory or nexus
add a dependency to the jar in your project pom
commit the pom into SVN
tell to your pears to check out from SVN the updated pom

You don't. In most circumstances you will use maven to deploy the project (the jar) to a repository manager (Nexus is by far the de facto for this).
Next you configure your co-workers to proxy their maven requests to yuor Nexus installation which itself "knows" about maven Central, thus providing a single point of presence for everything both from the outside world and published internally.
It's usually a good idea for another service such as Jenkins to be paired with a release plugin to execute the maven release procedure on your before, and for it to publish the artefacts too.
Here we have a long list of maven projects that our Jenkins installation builds upon noticing an SVN change. Upon the click of a button within Jenkins for the project, it can perform a complete build and release to our Nexus repository the project's newly built-and-tested artefacts.

How do i avoid the redundant "maven-update" in eclipse if i have 2 maven projects while one needs the other as a library

I have two java-projects configured with maven. One is a lib to the other. I always have to do maven install, maven update that my main projects gets the updated lib, when i change something.
Is there a way to avoid the annoying maven install / maven update procedure (in other words to automate this process)?

You can use the M2Eclipse plugin for Eclipse, which provides dependency management within Eclipse. If you have both projects open, it will automatically refer to the other project, without the need for mvn install. Any changes you make in one project will immediately be visible in the other - as long as your POM refers to the current version of the other project (usually SNAPSHOT).

If you develop them at the same time, It's probable that they shoudl be two modules of one project. If you set up them that way eclipse can manage their dependecy withou installing the lib into the repository.

Java, Eclipse, Ant, JUnit, Hudson, SVN, native libraries; can they all coexist?

What is the most proper way to accomplish all of the following:
Create a project in Eclipse
With an Apache Ant buildfile
That Hudson (or another more recommended CI system?) uses
And support for JUnit tests that are run by both Ant/Hudson and Eclipse
And check the proper folder structure into SVN so that future developers can replicate the build, and Hudson can automatically grab from SVN and run the Ant file to build/test/deploy the project
And the project has native libraries, if that makes any difference (I've already written an Ant task that can download and unzip the proper library files depending on the OS)
I already have my project with some source files and an Ant file, but I've been having trouble integrating it with Eclipse in an organized manner, so I would really love to start from a fresh Eclipse project, set it up correctly, and then copy my Ant file and my source files piece-by-piece into the project in the most Eclipse-compatible way.
I will be continuing to play around with everything in an attempt to get it working as I like it. But if you have experience with this sort of thing, perhaps at your workplace, please give as much information as you can.
My main goal here is to learn this once and use it in my future projects. For now, I am developing a client-server application consisting of a JOGL applet frontend (using JNLP files) and an unattended server app. Up until now I've been doing it all by hand: writing and building in Eclipse, dragging the applet jar into my FTP client, SSHing the server jar and restarting it by hand, and all with no testing process. I'm hoping that by the end, the build process will be something like this: test locally on my machine with a copy of the native libraries; commit code changes to SVN; Hudson svn updates, uses the Ant buildfile to compile and run all JUnit tests; if all the tests pass, it proceeds to copy the server jar to my dedicated server and restart the running server instance, and then copy the client jar to my web server.

When I start a new project, I generally take the following steps:
Create a new Java project in Eclipse, using the Java project wizard and opting to keep source and class files in separate directories ('src' and 'class')
Set up the project in Eclipse with the package structure you want
Create a new repository (or folder in your repository) for the project, with the normal /trunk/ /branches/ /tags/ set up
Using your SVN client, import the contents of the project folder Eclipse made for the project (at the level where the 'src' and 'class' directories are) into the trunk of the repository
Confirm everything is in Subversion correctly (check the trunk out to a new location)
Using Eclipse, delete the project (and the files on disk)
Create a new project using the 'Checkout projects from SVN' wizard, using the trunk of the repository
Create your Ant script, ensure the project builds correctly, and add the Ant script to the repository
Now, from Hudson:
Create a new job and set the Subversion URL to the root of the repository
Add a build set that will use Ant (I've always had to install my own version of Ant and ensure it's configured correctly to have this work) and will call the Ant script you
Use 'Build Now' to see if the job will build correctly
You can invoke your JUnit tests from Ant in the build script (first compile them with the javac task, then run them with the junit task). Hudson will track the test results if you ask it to.
You can use a shell script build step to copy your server jar to where it's needed, and restart the server instance. Like Mnementh said, it sounds like you've got the native libraries sorted...

If you are not tied to using ant, and are prepared to use Maven, then it is simply a matter of setting up Maven with the Eclipse plugin which generates the Eclipse projects for you.
Hudson already knows how to build Maven projects, so that is taken care of.
So long as you can pull your projects into eclipse, then it will be able to run the unit tests, and hudson can use the previously mentioned maven support to run the unit tests as well.
If you use Maven, then you will want to follow it's guidelines on how to create a project, here is a good starting point.
Hope this helps.

At our company we actually use Eclipse, Java, Ant, SVN, Junit and Hudson. That is all you mentioned except the native libraries. If you said your ant-buildscript already works with the native libraries that problem seems solved too. To integrate it well into eclipse you could do it in two ways: Use Ant also from Eclipse (has downsides) or the developer has to install the native library for his machine properly, so that Eclipse can compile without a problem and for continuous integration it will be downloaded by Ant.