We Keep Coding

Steps to change IntelliJ GitHub project to download and build source code for dependencies

Using IntelliJ IDEA, it is often a straight-forward task to check out a project from GitHub and get it to build locally.
Once the base project is imported from GitHub, the IDE will download artifacts which will allow the main project to run. The dependencies can be examined by using File > Project Structure... > Libraries.
So the IDE downloads dependencies to allow running, but not to build from source. The IDE is able to decompile classes, but the IDE will not automatically download the source code for those dependency libraries so that the programmer can alter the code.
This question is about the steps required in the IDE and project configuration such that a selected dependency will be built locally, and thus allow the programmer to alter the code.
What I tried was to import the project on which the main project was dependent, as a separate project, then configure the main project to utilize the local project instead of the downloaded artifact. The first step, downloading the sub-project and getting it to compile, was completed successfully.
So I ended-up with two projects, the main one, and the one on which the main project depended. The task at hand, if this was the appropriate way to get this done, would be to open the main project and take some action to convince the main project to use the local build, rather than the downloaded runtime "jar".
First, I edited the POM.xml to comment out the dependency for the sub project. Checking File > Project Structure... > Libraries, I could see that it was gone, and the build now failed (as expected).
I went to File > Project Structure... > Libraries > + (plus sign) and poked around with New Module, and Import Module, but I was not able to convince the original project to use the locally available sub project. The result from various attempts was that code in the base project was not able to import from the sub project (unable to compile).
What are the specific steps to take in the IDE to get what was a runtime dependency "jar file" to instead to build locally, and use that instead?

Use a SNAPSHOT version for the dependency (you'll need to change this in both your project's pom.xml and the dependency's pom.xml, so if the current version is 2.0.0 change it to 2.0.0-SNAPSHOT)
Then you can edit the dependency's code and run mvn install in the dependency to provide the new version of the dependency to your project.

TLDR: there is no simple and straight-forward way of downloading a project's code and the code of its dependencies to rebuild it in one go as a complete chain.
Your project depends on several other artifacts. From the screenshot, we may assume that this is a maven project, at least we can be sure that there artifacts with compiled classes available for download, because this happends during build. You can view the compiled classes of your dependencies, because Intelli has the capability of decompressing jars and decompiling code, obviously, but the contents you are viewing is read-only.
Sidenote: Maven convention is to create 3 separate jars for each project. One with compiled classes, one with source files only and one with generated documentation. By default intellij may not download these, but you can force it (right-click on pom.xml -> maven -> Download sources and documentation). This will attach the actual source code instead of decompiled classes to your IDE, so it's much easier to understand the code - but still, there is no option to modify it - it's still read-only extract from some jar.
So what if you want to actually edit the source? You have 3 options, all with its own set of problems that need human intelligence to solve:
You extract the decompiled source from classes jar
You extract the attached source from sources jar
You check out git repository of the dependency
Now, beware of the downsides of each approach:
You can be sure that the decompiled source matches your project dependency 1:1. But decompiled code is not easy to read, missing comments, etc. Also, some projects may not ship their build scripts with the classes jar. Anything more complex than mvn clean install may turn out to be a blocker.
You can be reasonably sure the code matches your project dependency, but this actually is not a given. There is a chance of human error, causing the sources to actually not match the compiled classes (build from different revision or whatnot). Much depends on the quality of the project, the discipline put into the build process and care to avoid environment specific configuration that is not part of the source. The larger and older is the project, the less chances are you are able to recompile it successfully using only src jar.
A sane man's approach. You should have your build scripts, readmes, tutorials, etc. Except, of course, if we are talking some obscure company internal project with zero effort put in its maintenance. Surely, there are the same issues as before: not all projects are rebuilt easily on any environment. There may be steps upon steps required for your workstation to be configured as expected. Hopefully, self-respecting open-source java projects are easy to build, but again - not a given - not all project are open-source, not all are self-respecting.
Important note: When checking out the git repo of your dependency - you must also make sure that you are using correct revision. If the project is maintained with respect for git tags/branches naming convention - you are in luck. Not a given by any means.
All the above is enough to discourage any attempts to automatically decompose dependencies to compilable units by your IDE, and all the burden is put into you. So let's assume the best - our dependency is a simple, self-contained java application that is easily built using simple mvn clean install. You have it checked out in a separate project in your IDE. You identified correct git revision that matches version your project depends on.
Now let's apply your little change and test it. First thing you want to do is change pom.xml of your project to use a made up version of your dependency. It should be a -SNAPSHOT version for clarity and tidiness. You may of course build your modified dependency with real release version - but please be wary of how maven manages dependencies. If you install version 1.0 yourself - it stays in your local repo forever. You will forget about it, and will be using your fake 1.0 version when building all other dependent projects unless you manually locate and remove it from repo. So stick to 1.1-SNAPSHOT.
Now every time you need to apply a small fix to your dependency, execute mvn clean install in its repo, then make sure your actual project depends on the correct new SNAPSHOT version, execute your maven clean install and that's it.
Note that all this has very little to do with Intellij. You are not expected to modify any library paths, advanced project settings, or links to jars. Stick to modifying pom.xml and you are set.

Eclipse settings for maven modules

I cloned the maven project to my local disk. The project has 3 modules as well as a parent project. i marked the 3 modules with the 3 colors-- the 4th, blue color is the project itself.
The project is running all fine on maven both on Eclipse and on command line.
When i do a mvn install on Eclipse, the project is built and all the tests are run successfully -- i see the good old BUILD SUCCESS report of maven on Eclipse console.
The problem is -- the modules are all messed up. (this "messed up" is the best way i can put it for now.) Eclipse doesn't recognize the dependencies between modules and gives checked errors to some of the code that use outside dependencies.
At one place-- lombok #Getter is used in one class. import lombok.Getter; is seen by the compiler-- no checked error to that. but eclipse doesn't recognize the getter declaration of lombok, say getField() for the field field. i know this because : a) in the class itself-- gives a warning that .. the private field field isn't used.
b) gives a checked error to getField() a reference from another class.
Thought this might be an interesting example to how the dependencies are "working". i'm guessing one dependency that lombok is using is intercepted somewhere along the way to give this inconsistency.
Following is the project root directory structure on disk. (the 3 colors are the 3 modules, have to hide the names for confidentiality):
the 3 modules are defined in the pom.xml as follows:
i see the following in Eclipse project explorer:
so in the Project explorer, the 2 modules/tests (green & yellow) that have java dir, i.e. src/main/java appear twice. the orange module has only src/main/resources and no src/main/java.
In the second occurrences of the modules in the project explorer (the image above), there are no dependencies of any kind, not even Java itself and hence all those red dots all around (compiler errors). The first occurrences of modules have Referenced Libraries and no Maven Dependencies:
How to fix this project so that i can code and run in Eclipse? Just to pin down-- project configuration is all fine and it runs successfully on maven. the problem is Eclipse's own settings on it.
so far i tried the following:
made the project faceted and added Java as the only one to its facets
imported as Maven project -- File > Import > Maven > Existing Maven Projects
ran mvn eclipse:eclipse on its pom.xml
Wasn't even seeing the Build path before these.
i'd appreciate the help-- wore me out.
everyone else on the project is IntelliJ developer-- no workspace to compare with.
i've seen Importing Maven project into Eclipse and some other useful discussions.
i'm not sure whether i should post .project, .settings and some other file contents also.
will do if necessary.
TIA.

Make sure to not only import the parent pom.xml, but also the Maven modules as projects into Eclipse. To do this in one go, use
File > Open Projects from File System... or
in the Git Repositories view right-click the root folder and choose Import Projects...
Make also sure Lombok has been installed into Eclipse, otherwise Eclipse's own incremental compiler would not generate e.g. getter and setter methods that are defined by Lombok annotations. See the Lombok documentation of how to install Lombok into Eclipse. If you have installed Lombok after importing the projects, it might be that you need to do a Project > Clean... or close and reopen again the projects (via the right-click menu).

How to build Java project using IntlliJ IDE instead of via shell?

I cloned a GitHub repo onto my disk an successfully built the project by:
ant clean-all resolve create-dot-classpath
within the projects root directory.
I assume this should also be possible from IntelliJ IDE but that fails by doing nothing at all - it simply states "All files are up-to-date" ... what I do is:
"Make module 'pentaho-kettle'"
"Compile module 'pentaho-kettle'"
I could use a hint where to look for a solution or what configuration is most likely missing.

As the project is a multi module project based on Ant and Ivy, you have to
import the project from sources
configure modules (like core, ui, the plugins, …) with the right directories for main and test sources and resources (they seem to overlap with java sources in that project)
setup the class path using an Ivy plugin (I recommend IvyIDEA), use the context menu on the ivy.xml file
and have luck to build. It will be some try and error!
You may also have a try importing from Eclipse as those .project and .classpath files are checked in. But than you also have to check the module directories, the import is more a partial guess and I worry that IntelliJ will not get any information about Ivy from the Eclipse setup.

How to remove Modules from a Intellij Maven Project permanently?

I am currently working on a larger scale Maven-based project in IntelliJIdea 12.1.6 Ultimate. I have been working with IntelliJIdea since about 5 months.
An included module has dependencies on another module. The dependent module's source was also part of my project until recently. Since I removed the dependent module from my project, I get compile errors whenever I am trying to compile the source without maven.
The pom.xml of removed modules in Intellij seem to be placed onto the Settings->Maven->Ignored Files. I cant seem to remove it from there, only check or uncheck it. It's not possible to include the module again since IntelliJ will say its still under Ignored Files.
2 ways allow me to compile again: Uncheck the pom from Ignored files, which will include the module again in my project. Or delete the source of the dependent module, so my project will load the dependent module from the maven repository. But whenever I update my project from svn, the source of the dependent module is restored (I don't know why this even happens since its not part my project) and the cycle begins anew.
I googled this for a while since it gets really annoying. It became a problem with several excluded modules. I could rebuild the intellij-project from scratch but since a lot of IntelliJ settings were made (not related to the problem) I would rather solve this.
Any help is appreciated, I guess I must be missing something

A collegue who has more in-depth knowledge of maven told me the answer:
The trick is not to remove the source module from the intellij project but to remove it from maven (in the maven projects tab in intellij). Intellij will ask then to remove the source module also from the intellij project and its finally gone.
Right click on the maven project -> remove projects
Would not have guessed this makes such a difference.

Do any of these two points work for you:
NEW there might be dependency in one of your sub-modules pointing to the removed artifact. To make sure it's the situation, you may want to rename your $HOME/.m2 and rebuild the project. If this is indeed the case, just search for the dependency in your poms and delete it from sub-modules
right click on the parent pom.xml -> Maven -> Reimport
copy the project into a separate dir. File -> Import Project -> Choose newly created dir -> Choose Maven?
PS. Idea is excellent in maintaining the project structure in accordance with Maven project. Once you make a change to your pom, you need to reload the project. Idea can also automatically detect changes made to your pom.xml and apply them to project. To enable this, press Ctrl+Shift+A, type 'maven auto', choose "Importing", checkbox "Import Maven project automatically";

This may not relate directly to the OP, but...
Using IntelliJ 2016.5, to remove a Maven module and it's underlying source code and remove it as a sub-module from the parent, go to the Project Files tab, find the unwanted module and use the triangle symbol to open it, then delete the entry that represents it on disk.
Hitting Delete on the module itself will only mark it as hidden within the IDE.

Build single class in IntelliJ using dependencies previously built by maven

I have a application using a maven build and am using IntelliJ IDEA. Both Maven and IntelliJ build classes into the same target folder, however whenever I choose to rebuild a single class in IntelliJ it complains that it cannot find it's dependencies, even if maven just built them and they are in the output folders. If I do a full rebuild in IntelliJ it will build the class.
How do I make IntelliJ use the classes maven built as dependencies for the single class it is building?

If it's purely class files, I'm not sure why it'd be complaining if they're all under the directory you have configured IntelliJ to use for the project's compiled classes.
More frequently, what I've run into is the build creates some things other than class files and you have to create a library that points at that other stuff so that IntelliJ knows about them. Another wrinkle are resource files as those are hunted for relative to the project's working directory which is, by default, usually the root directory of the project which is rarely correct as most resources are dumped under a "build" or "deploy" directory.
If all else fails, you should be able to spit out verbose Maven build info and compare the classpath it's using at compile time to the one that IntelliJ uses to try and figure out where they differ.