What is the exact dependency I need to develop a Gradle Plugin in Java? Ideally I would like to get it from a well-known repository such as Maven Central or similar.
I have a Maven project with a core functionality and I just added two extra plugins, one for Ant, one for Maven. They are already tested and working; easy! Now, I wanted to add a third module for a Gradle plugin to make this functionality also available from any Gradle project.
However, I can't find the exact dependencies I need to develop a Gradle plugin.
The Gradle docs (such as https://docs.gradle.org/current/userguide/java_gradle_plugin.html) are not very well written to say the least. They mention:
the gradleAPI() dependency
or the java-gradle-plugin dependency
But they are quite unclear... no group, no version (really?).
If anyone can enlighten me to where I can get these dependencies from, I would be very thankful.
Gradle's public and internal APIs, aka gradleApi(), are bundled with the Gradle distribution and not independently published and therefore not easily consumable by Maven builds. There's the pending epic #1156 (Ensure plugin cross-version compatibility by allowing a user to depend on gradlePublicApi()) that might help here.
Since Gradle plugins are best to be built with Gradle, a pragmatic solution is to invoke the Gradle build from Maven and attach the produced artifact to the Maven build. Andres Almiray (aalmiray) once described this in the blog post Running Gradle Inside Maven (Web Archive Link). He describes the following high level steps:
Create a new Maven module (e.g. gradle-plugin) and add attach it to the parent POM
In the POM of gradle-plugin add a dependency to your core module. Use the maven-dependency-plugin to store dependencies to the Maven build folder, e.g. target/dependencies.
Create the build.gradle, add a Maven repository that points to target/dependencies (step 2) and let it depend on the core module as well as gradleApi(). Implement the Gradle plugin.
Use the exec-maven-plugin to invoke the Gradle build.
Use the maven-resources-plugin to copy the Gradle built plugin jars to the standard Maven build folder.
Use the build-helper-maven-plugin to attach the copied jars to the Maven build.
Sample project to be found here (gradle-in-maven).
https://docs.gradle.org/current/userguide/custom_plugins.html#sec:custom_plugins_standalone_project
In here it is mentioned that it is gradleApi() and I know that this works (from experience). The localGroovy() on that page is only needed if your plugin code uses groovy (does not apply if you only use groovy in the build.gradle of your plugin).
java-gradle-plugin is a library that makes it a bit simpler to make plugins, it is not required though. I personally prefer using gradleApi only.
EDIT:
It appears I've misunderstood the question. Here are the steps to get gradleApi jar:
Create a Gradle project with your desired Gradle version.
Add implementation gradleApi() dependency.
Import/run the project once.
Go to your .gradle folder (located in home folder in Linux-based operating systems).
Open caches folder
Open the version folder you want, e.g. 6.0.1
Open generated-gradle-jars folder.
Copy the jar to wherever you want and use it.
For me the 6.0.1 jar is at ~/.gradle/caches/6.0.1/generated-gradle-jars/gradle-api-6.0.1.jar
Please note that I have not tested this, I know the jar is there but I haven't tried using it.
I have a simple testing-purpose gradle project, which I want to scan its dependencies using gradle dependencies command. Before I do so I want to make sure that the project's dependencies are actually found in the gradle's cache (.gradle/caches/modules-2/files-2/1). To do so I run the gradle assemble command to download the missing dependencies before scanning.
I found out that its working only if the project has a src/main/java folder with a Java file inside it (even if that Java file is completely empty).
Is this a valid workaround? Is there any better solution to guarantee the dependencies are found in the cache folder before scanning them?
What is the reason that you want to do that?
assemble task assemble your source files, if there is nothing to assemble the task is not needed to run. The fact you are adding the java file to src its a hack to run this task and its children tasks.
Depending on what you want to achieve there are few ways to 'scan' dependencies.
For more info you can visit https://docs.gradle.org/current/userguide/userguide_single.html#sec:listing_dependencies
Aditionally:
There is a netflix plugin that I believe can scan through your gradle scripts a check unused dependencies https://github.com/nebula-plugins/gradle-lint-plugin
There is a plugin that can scan the vulnerabilities of used dependencies etc https://jeremylong.github.io/DependencyCheck/dependency-check-gradle/
I am upgrading a project from Ant to Gradle. The project uses a aqapi13.jar(this is oracle aq jar. This is needed as the project reads from an oracle-queue and writes to an activemq queue.)
The ant project contains the jar aqapi13.jar in the libs folder. But iam trying to get this dependency from a repository instead of having it in the libs folder.
However, iam not able to find a repository which contains this jar. All the repositories that I have seen contain aqapi13-9i.jar, but not aqapi13.jar.
Anyone knows the difference between aqapi13.jar and aqapi13-9i.jar and how to get the needed aqapi13.jar from a repository.
Advance Thanks
Some dependencies are never found in public repositories because of the license they have.
One way to use this dependencies is to create your own repository (e.g. artifactory, nexus, archiva). Then you are free to put in every artifact you want (as long as you do not publish the repository). This repository can also serve as a mirror for maven.
Another way could be to mark this dependency as system scope.
I am having a java project with a ant build file, using this ant file i create an ejb of the project and deploy it on the jboss server.
Now I am planning to use maven and convert this existing project which consist of nearly 28-30 jar's in its class path(jars related to ejb3, hibernate, jboss, etc).
I can easily do it using eclipse i.e right click project goto maven and click Conver to Maven.
A pom.xml is generated and the MavenClassPath Container is also added to the project.
Now I want to know how to get rid of those 28-30 jar's present in the lib folder of the project and in the classpath. i.e. I want my pom.xml handle all the dependencies.
Does Maven provide any mechanism to achieve this goal while converting the project or I have to add all of these jar dependencies one by one manually in the pom.xml file.
The intention of doing this is I want to have common maven remote repository where the jars will be stored and each developer machine will point to it through their maven project.
Thanks
I think you're after a repository manager like Nexus (I use Nexus, it seems to be the most popular http://nexus.sonatype.org/ ).
Nexus can be used as:
A proxy repository (for Maven Central, etc)
A repository for your own releases.
Nexus provides user management for your developers to release builds into the repo.
Developers will then point their Maven settings.xml file to your Nexus repository, and all their dependencies will come from here (Nexus will cache them).
I'm afraid you will have to configure the dependencies individually, but that is a good thing, because you should pay attention to what version ranges you are interested in for each dependency.
Any jars which can't be found in Maven Central, etc, you can add to your own Nexus repository .
Ofcourse there are alternatives to Nexus, but I haven't used any.
HTH
The most important thing i can recommend is to use a Maven Repository Manager (Nexus, Artifactory or Achiva or other..).
Second your pom conversion via Eclipse shows me that you are not using an up-to-date Eclipse nor an up-to-date Maven Plugin for Eclipse. The best thing would be use Eclipse-Indigo (m2e is the newest and greatest).
Furthermore you have to go through all your jar's and add them step by step to you pom (dependencies) and see if your project can be compiled. This should be checked on command line not inside Eclipse.
After you got a working pom.xml file put it into your version control and check if you can remove some of your added dependencies based on transitive dependencies. After that you can finally delete your lib folder.
When I open a POM file and click on the "Dependency Hierarchy" tab at the bottom, it gives me the error, "Project read error". It works with other projects in the same workspace, just not with this one. Any ideas?
EDIT
In response to #Yhn's answer.
Running the compile and package phases outside of Eclipse from the command-line work as expected. It compiles the application and builds the final WAR file.
Eclipse is indeed pointing to the default location of the Maven settings.xml file, so it should be aware of the custom repositories that are defined in it (my company has its own Maven repository).
I can open and edit the POM file from Eclipse, so it must have read/write permissions to the file.
The project is not configured in Eclipse as a Maven project, so I cannot run the package phase from Eclipse (I can only run it from the command-line).
I wonder if it has anything to do with the fact that I was having trouble building the project with Maven 3 because apparently some of the transitive dependencies are configured for Maven 1, which Maven 3 does not support (this is my theory anyway, based on some of the error messages). I can build the project with Maven 2, but I still get messages such as the following:
Downloading: http://dist.codehaus.org/mule/dependencies/maven2/org/codehaus/xfie/bcprov-jdk14/133/bcprov-jdk14-133.pom
[INFO] Unable to find resource 'org.codehaus.xfire:bcprov-jdk14:pom:133' in repsitory mule (http://dist.codehaus.org/mule/dependencies/maven2)
It must be able to find these dependences however, because it downloaded the JARs just fine and can build the application. It seems like the problem is that the dependencies don't have POM files associated with them, which is maybe why they cannot be used with Maven 3. This might also be why I cannot view the Dependency Hierarchy in Eclipse.
EDIT 2
I converted the project to a Maven project by going to "Configure > Convert to Maven Project". When I open the POM file, I see the error:
ArtifactDescriptorException: Failed to read artifact descriptor for woodstox:wst (Click for 140 more)
(woodstox:wst is another transitive dependency of the project). An error appears in the "Markers" view for seemingly every depedency and transitive dependency in my project. However, I can successfully build the project by doing a "Run As > Maven build". (Edit: This might be because this project has no Java source code, but the JARs of the dependencies correctly appear in the final WAR.) The Dependency Hierarchy still gives the same error--"Project read error".
About the "Unable to find resource" messages--but this only appears for a handful of transitive dependencies. The project has many more transitive dependencies, but these messages do not appear for them. It seems like, because the dependencies do not have POM files, that Maven tries to search for them every time the project is built. Is this normal not to have POMs??
How might I go about getting a repo manager? Is this something that would have to be installed on the company's Maven repository or can you install it on your own workstation?
I had this problem with some non-maven jars that I needed to include in my maven project. I put the jars in my local repository using this maven command:
mvn install:install-file -Dfile=/test/gov.nist.xccdf-1.2.jar -DgroupId=gov.nist -DartifactId=xccdf -Dpackaging=jar -Dversion=1.2
Then I referred to them as dependencies in my pom.xml:
<dependency>
<groupId>gov.nist</groupId>
<artifactId>xccdf</artifactId>
<version>1.2</version>
<type>jar</type>
<scope>compile</scope>
</dependency>
This worked fine for a while, but I must have upgraded something in eclipse, and I started getting the error you describe.
The fix is to take them out of the local repository, and redo the install-file asking maven to generate a pom for you:
mvn -DgeneratePom=true install:install-file -Dfile=/test/gov.nist.xccdf-1.2.jar -DgroupId=gov.nist -DartifactId=xccdf -Dpackaging=jar -Dversion=1.2
The command will cause a pom to be generated in the same directory (within your local repo) where the jar is placed.
In recent versions of eclipse maven support, the maven build is still done with maven 2 (if you right-click your pom.xml, choose run as mvn package for example). The dependency analysis is now done with an embedded maven 3 plugin, which doesn't like a dependency that has no pom.xml.
I had the same problem. If you made your project a maven project, you should run:
Right Mouse Click on project | Maven | Update Dependencies or
Right Mouse Click on project | Maven | Update Project Configuration
That worked for me.
Given the information through the comments:
mvn compile/mvn package work through console (I'm assuming outside of eclipse). Based on that, the POM file should be correct and dependencies can be resolved.
However; it could be that eclipse's settings aren't correct. For example, if eclipse has some own maven properties (instead of the default ~/.m2/settings.xml file); I can imagine it to fail resolving dependencies configured in the settings.xml used by maven itself. The settings for this should be in the Eclipse Preferences # Maven > User Settings.
Also make sure that Eclipse can actually read the file (not locked, correct rights, etc).
Should this seem to be correct; can you try to run the mvn goal compile from eclipse? You can do this by right-clicking the project and selecting run as > maven package. That way it should run maven with the same settings as eclipse is using, and might show any additional errors in it's configuration.
--
In response to additional information:
Basically it tells you - when using Maven 2 - that it can't find a dependency resource (in this case bcprov-jdk14) in the given repository (codehaus/mule). When I search that maven module (bcprov-jdk14) I find it in the Maven central repo (through our local nexus) # repo1.maven.org/maven2. Maybe that causes the error you're getting with Maven2?
And as the Maven 3 doc says; for Maven 1.x repo's; you should proxy them through a repo manager that can serve it to you as a maven 2 repository (I believe we do that here too for some repo's through Sonatype Nexus OSS)
ps.: Enabling maven dependency management in eclipse should be enough to be able to use maven run targets; if you would want to test it.
--
Sound like it can be multiple problems... given that the normal maven build works fine, one would think that maven can at least get the artifacts it depends upon and their dependencies... Yet, eclipse doesn't seem to be able to.
Double check if your Eclipse internet connection settings are correct? Since you were talking about a company repo, I'm assuming there will probably also be a proxy (I had to set up my Eclipse to use that one too; though for plugins (see next point)).
Another possibility, from experience, can be a rule-based proxy block on *.pom url requests (our proxy here blocks that to force usage of the local repo manager, how evil and annoying that is :(). You might want to try and open the .pom file it tries to download manually in your browser to see if this is the case (I can't, for example...).
To get the repo set up, you would probably have to ask the one responsible for the repository to add it as a maven2 proxy.
At the very least, it seems Eclipse is having issues getting the dependency poms (which are needed to build the dependency tree) from the internet. You could always try to ask some ICT crew (if they are capable enough...) about it at your company, maybe they have some useful hints.
right click on the project, Maven->Update Project->Tick "Force Update of Snapshots/Releases"
I had the same problem when I added a new dependency to pom.xml without network connection. After this, I had so many problems such as "Missing artifacts", "Missing Descriptor", or "Project Read error". I solved this
delete the bad dependency just added from local repository.
rebuild local index.
re-add the dependency
<properties>
<!-- ********************** -->
<!-- Plugin's properties -->
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<java.version>1.7</java.version>
</properties>
this pom setting worked for me
Usually this happens if there is a typo in the pom that Maven cannot understand and then it throws Project read error. Like in my case I copied a maven dependency snippet and I by passed the version. Instead of typing the actual version # it was getting it dynamically ${asciidoctorj.version} and my pom didn't have this in the properties section. So I got this error. Once I removed that and put a version number it started working.
I had this after copying a local repository folder from another machine. The fix was to go into the repository folder I copied and delete
_remote.repositories along with the 'LastUpdated' files then refresh the Maven dependencies in Eclipse (Alt+F5)
I faced same situation today. In my case it's caused by dependency name in wrong case. E.g.
Project A -> Project B -> Project C
In project B's pm file, I mistakenly specified the dependency artifacts name with "c" in stead of "C".
As in mac os, the files system is case insensitive. so I can build it in command line without detecting this mistake.
In Eclipse it breaks, but it provides very bad error message. In the error list, it says "dependency problem", but the name of dependency is empty. It also can't identify which line in the POM causes the problem.
When try to open "Effective POM" in POM viewer, it will show project read error. In the pop up error dialog box, it will show:
Could not read maven project
java.nio.channels.OverlappingFileLockException
All the messages are useless and misleading.
I finally detected this problem by submit to Jenkins CI after wasted hours of time.
I did maven clean install and the errors were gone
I faced the same issue of "Project read error" while clicking on "Dependency Hierarchy" tab. In my project I had a parent pom and child pom.
Parent pom had a property <appVersion>4.5<appVersion> that was being used in child pom
<version>${appVersion}<version>.
I selected the main project, right click and choose run as maven clean. The console display showed warning that version is using expression but should be a constant. Replacing ${appVersion} with constant value of 4.5 fixed the issue.