I want to build a java project with several dependencies in gradle, some are jar dependencies from the maven central repository, some are other java project dependencies, but there is also a native JNI dependency. I've managed to build the native dependency with gradle as a seperate project.
Now I want to import the compiled .so library as a depedency in my java project. I thought it would work something like this:
dependencies {
compile project(':nameofnativedependency')
}
The project is also listed in my settings.gradle file. However, when I compile the project, I can get an error:
Could not determine the dependencies of task ':test
Configuration with name 'default' not found.
So something must be misconfigured. What is the right way (I hope there is one) to compile a native project and import the .so file in the calling project?
The Gradle native plugin uses the new rule based software model. Model elements are not instantiated until after build.gradle has veen processed, and once instantiated they are immutable. To access those elements, you need to put your code inside the components closure, and access model elements through the $ variable (bearing in mind that elements accessed that way will be instantiated, therefore immutable).
To integrate a java build with a native build, you will probably want to use the java software model plugins, instead of the legacy java plugin. I have been playing around with it, and have a buildscript that I'm starting to feel a little proud of, incorporating all of the steps from generating headers from class files, building libs, copying into src/main/resources to be bundled in the jar, then extracting and loading libs at runtime (though this part's not quite done yet; Scijava's native library loader is probably the way to go).
Related
Using IntelliJ IDEA, it is often a straight-forward task to check out a project from GitHub and get it to build locally.
Once the base project is imported from GitHub, the IDE will download artifacts which will allow the main project to run. The dependencies can be examined by using File > Project Structure... > Libraries.
So the IDE downloads dependencies to allow running, but not to build from source. The IDE is able to decompile classes, but the IDE will not automatically download the source code for those dependency libraries so that the programmer can alter the code.
This question is about the steps required in the IDE and project configuration such that a selected dependency will be built locally, and thus allow the programmer to alter the code.
What I tried was to import the project on which the main project was dependent, as a separate project, then configure the main project to utilize the local project instead of the downloaded artifact. The first step, downloading the sub-project and getting it to compile, was completed successfully.
So I ended-up with two projects, the main one, and the one on which the main project depended. The task at hand, if this was the appropriate way to get this done, would be to open the main project and take some action to convince the main project to use the local build, rather than the downloaded runtime "jar".
First, I edited the POM.xml to comment out the dependency for the sub project. Checking File > Project Structure... > Libraries, I could see that it was gone, and the build now failed (as expected).
I went to File > Project Structure... > Libraries > + (plus sign) and poked around with New Module, and Import Module, but I was not able to convince the original project to use the locally available sub project. The result from various attempts was that code in the base project was not able to import from the sub project (unable to compile).
What are the specific steps to take in the IDE to get what was a runtime dependency "jar file" to instead to build locally, and use that instead?
Use a SNAPSHOT version for the dependency (you'll need to change this in both your project's pom.xml and the dependency's pom.xml, so if the current version is 2.0.0 change it to 2.0.0-SNAPSHOT)
Then you can edit the dependency's code and run mvn install in the dependency to provide the new version of the dependency to your project.
TLDR: there is no simple and straight-forward way of downloading a project's code and the code of its dependencies to rebuild it in one go as a complete chain.
Your project depends on several other artifacts. From the screenshot, we may assume that this is a maven project, at least we can be sure that there artifacts with compiled classes available for download, because this happends during build. You can view the compiled classes of your dependencies, because Intelli has the capability of decompressing jars and decompiling code, obviously, but the contents you are viewing is read-only.
Sidenote: Maven convention is to create 3 separate jars for each project. One with compiled classes, one with source files only and one with generated documentation. By default intellij may not download these, but you can force it (right-click on pom.xml -> maven -> Download sources and documentation). This will attach the actual source code instead of decompiled classes to your IDE, so it's much easier to understand the code - but still, there is no option to modify it - it's still read-only extract from some jar.
So what if you want to actually edit the source? You have 3 options, all with its own set of problems that need human intelligence to solve:
You extract the decompiled source from classes jar
You extract the attached source from sources jar
You check out git repository of the dependency
Now, beware of the downsides of each approach:
You can be sure that the decompiled source matches your project dependency 1:1. But decompiled code is not easy to read, missing comments, etc. Also, some projects may not ship their build scripts with the classes jar. Anything more complex than mvn clean install may turn out to be a blocker.
You can be reasonably sure the code matches your project dependency, but this actually is not a given. There is a chance of human error, causing the sources to actually not match the compiled classes (build from different revision or whatnot). Much depends on the quality of the project, the discipline put into the build process and care to avoid environment specific configuration that is not part of the source. The larger and older is the project, the less chances are you are able to recompile it successfully using only src jar.
A sane man's approach. You should have your build scripts, readmes, tutorials, etc. Except, of course, if we are talking some obscure company internal project with zero effort put in its maintenance. Surely, there are the same issues as before: not all projects are rebuilt easily on any environment. There may be steps upon steps required for your workstation to be configured as expected. Hopefully, self-respecting open-source java projects are easy to build, but again - not a given - not all project are open-source, not all are self-respecting.
Important note: When checking out the git repo of your dependency - you must also make sure that you are using correct revision. If the project is maintained with respect for git tags/branches naming convention - you are in luck. Not a given by any means.
All the above is enough to discourage any attempts to automatically decompose dependencies to compilable units by your IDE, and all the burden is put into you. So let's assume the best - our dependency is a simple, self-contained java application that is easily built using simple mvn clean install. You have it checked out in a separate project in your IDE. You identified correct git revision that matches version your project depends on.
Now let's apply your little change and test it. First thing you want to do is change pom.xml of your project to use a made up version of your dependency. It should be a -SNAPSHOT version for clarity and tidiness. You may of course build your modified dependency with real release version - but please be wary of how maven manages dependencies. If you install version 1.0 yourself - it stays in your local repo forever. You will forget about it, and will be using your fake 1.0 version when building all other dependent projects unless you manually locate and remove it from repo. So stick to 1.1-SNAPSHOT.
Now every time you need to apply a small fix to your dependency, execute mvn clean install in its repo, then make sure your actual project depends on the correct new SNAPSHOT version, execute your maven clean install and that's it.
Note that all this has very little to do with Intellij. You are not expected to modify any library paths, advanced project settings, or links to jars. Stick to modifying pom.xml and you are set.
I have used SBT to build a pure-java project. This project consists of multiple modules, and projectB depends on projectA.
Now after switching to JDK 11, this dependency should be modeled not via classpath but via module path.
Is it possible to tell SBT that module depencencies must be resolved via module path instead of the classpath?
Basically I have set up a inter-(sbt-)module depencencs that says projB.dependsOn(projA) - so can I tell SBT somehow to put this depencency on the javac module-path instead of using the classpath? I see currently no way of doing that.
EDIT: Further analysis showed that I needed also to use normal library depencencies (from libraries on a nexus repository) as module depencencies. So the next question would be - how can I treat these library depencencies as module depencencies in SBT?
I do know that SBT is a scala build tool, and that it is something of a stretch to use it for a java-only project, but I use SBT for other (scala-)projects, and I do not want to introduce too many different build tools, until I really have to. So - is there a sbt-level solution for this?
EDIT: To clarify the issue - the project in question is a kind of open-source library which we modified. The SBT build file is from our organization, so I have to maintain it.
The project switched to using module-info classes, and it depends on other (binary-only) libraries via module path, if you include module-info.java in the compilation, that is.
So in our SBT build I have to use the module path for some of the external library depencencies.
Also the SBT-subprojects have dependencies on each other, i.e. we also have to use the module path for that.
Is there a way to tell SBT that:
This library must be on the module path, and not on the classpath
This project dependency must be on the module path, and not via classpath?
That is the core of the question. I may be able to write a plugin for the first problem (having not libraryDepencencies, but modularLibraryDepencencies for example) - but I do not yet know how I could handle the second part (tell a project that project depencencies must be on the module path and not on the class path).
I am developing a Java application using IntelliJ Idea 14.1.4.
If it would have been solely Java application, I would have known exactly how to structure the project in Idea:
A single Java project, containing several modules: One for each part of the application (JAR).There will be at most 4-5 JARs.
The dependencies between the modules are also known: Protocol does not depend on anything, everything else depends on Infrastructure, and so on.
Next, I would like to use Gradle scripting for managing the project. So my question is what is the best practice to structure the code in Idea?
Should I create a single Gradle Project and a Gradle module for each of the JARs?
Should I create a single Java project (or maybe empty project) and Gradle modules for each of the JARs?
Should I create a single Gradle project and each of the modules will be a Gradle's sub-project? Maybe it will be better to have an empty project and several Gradle modules because not all of the JARs are closely coupled?
Since I have never used Gradle before, I would appreciate any guidance for the best practices when combining both Gradle and Idea.
Thanks,
Guy
As long as possible, I would keep the code in one source repository. On the root, I would have an "empty" project not outputting anything. All your jar projects would be sub-projects (in Gradle terms). You include them via the settings.xml file located in the root project.
Each sub-project has its own build.gradle file. In those files, you can easily define the dependencies between your sub-projects, e.g.:
dependencies {
compile project(':subProject3')
}
For convenience, I often create a special export task to put all artefacts in one export/ folder on the root level so that you don't need to go through all those sub-folders to get your stuff.
task export(type: Copy) {
from project(':subProject1').jar
from project(':subProject2').jar
from project(':subProject3').war
into 'export/'
}
IntelliJ Ultimate 14 works fine with this approach. You can simply hit Make to compile everything. You might also want to configure your project settings to run gradle jar or gradle export during a make if you prefer.
I've seen a lot of projects, even from big companies like Elephant Bird (Twitter) and Akela (Mozilla) that offer source and ask you to compile it yourself instead of also offering jars. Is there some benefit to compiling in your own environment instead of just downloading a jar someone else has compiled?
Dependencies are not in the same location or even have the same version on every machine. It is simpler to detect where they are at compile-time.
If there is any native code (sometimes just for optimization) in a project, there are probably platform-dependent flags that need to be set at compile-time.
The short answer is dependency management. Most public OSS Java projects offer jars by publishing them to Maven Central. You are expected to use a build system like Gradle, Ivy, or Maven to manage your dependencies - these tools will automatically download the library you want along with any of its dependent libraries and be smart about it, caching it on your local filesystem so if a library is shared across multiple libraries it won't be downloaded twice.
As for the example projects you listed, Elephant Bird is available via Maven Central whereas Akela tells you exactly how to create your own jar (perhaps it's not quite far along enough to justify going through the rigmarole of publishing to Maven Central):
Building
To make a jar you can do:
mvn package
To make a Hadoop MapReduce job jar with no defined main class in the manifest:
mvn assembly:assembly
Without an automatic build system its hard to maintain a current version of the jar file online. Including the jar file in the repository is generally not a good idea as users who clone it don't need the compiled jar, they want the code. So unless the publisher explicitly adds a jar file to a download location outside of the sourcecode repository and updates this file every time the application changes you have to compile it yourself. Automatic Build systems can help a publisher to provide a current compiled jar to it's users but for smaller projects it's not always sufficient to go through the trouble of setting one up.
Gradle is an excellent dependency manager. However, there does not seem to be an obvious way with the Java plugin to pull the source artifacts from an entire dependency tree.
Building with GWT requires .java source files, not .class bytecode files.
When declaring dependencies among my own Gradle projects, I can define a custom Gradle configuration and extend the Jar task type to produce a custom source artifact. From the parent project, I can then easily rely on the subproject's sources in a loosely-coupled way.
But what if my subproject then relies on some non-source dependencies from mavenCentral()? How do I get the parent project to pull the sources from those transitive dependencies (the whole tree), assuming they are available?
There isn't an easy way to resolve a configuration's source Jars at the moment. (You'd essentially have to reimplement what the eclipse and idea plugins do). The next Gradle version (1.12) will provide a new artifact resolution API, which will make this a lot easier.