We have a situation where we need one application to be able to connect to to two versions of kafka(0.7.2 and 0.10.0+) and act as a router. I'm trying to omit using two runtimes here as we need this to be stupid fast, so want to prevent additional serialization/deserialization when sending data between runtimes.
To do this, i've tryied to repackage the old kafka driver from package kafka to old.kafka like so:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>kafka-router</artifactId>
<groupId>org.deer</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>old-kafka</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<kafka.version>0.7.2</kafka.version>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>3.1.1</version>
<executions>
<execution>
<id>unpack</id>
<phase>compile</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.9.2</artifactId>
<version>${kafka.version}</version>
<type>jar</type>
<overWrite>false</overWrite>
<outputDirectory>${project.build.directory}/classes</outputDirectory>
<includes>**/*.class,**/*.xml</includes>
</artifactItem>
</artifactItems>
<includes>**/*.java</includes>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>true</overWriteSnapshots>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>kafka.</pattern>
<shadedPattern>old.kafka.</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
I'm using dependency plugin to unpack kafka classes to target/classes & shade plugin to repackage them. The reason for this is that the final jar should act as if it is a kafka driver jar(it has no other transitive dependencies, therefore it can't cause some mismatch using kafka instead of old.kafka. But that's not really the point here, just trying to prevent out-of-topic questions.
The main problem here is that when i look at the jar that has been installed to .m2, it looks correct(having the old.kafka package):
But when i try to use this jar as dependency like so ...
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>kafka-router</artifactId>
<groupId>org.deer</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>router-app</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.deer</groupId>
<artifactId>old-kafka</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
</dependencies>
</project>
... and reference it in a class like so ...
package org.deer.test;
import old.kafka.producer.ProducerData;
public class TwoKafkaDriversExample {
public static void main(String[] args) {
new ProducerData();
}
}
... the import it self is not working. I have a suspicion that the shaded jar is missing something maven-related, but haven't noticed anything. Another possible though is that shade plugin or asm doesn't like the bytecode that scala classes are generating.
Ok, so i've been able to figure this out. The import error is a problem with intelij, for some reason it doesn't see the repackaged classes. But the maven does, with using correct constructor and adding scala-lang dependency(it was complaining about missing Seq class) i was able to build this.
Full example uploaded on github - https://github.com/Marssmart/kafka-router
Related
When I build the maven Module that is using Antlr4 and look in the Jar the compiled generated sources from Antlr4 are in the Root folder.
But when I want to include the Jar with the generated sources in my Antlr4-impl module as dependency where I want to use the generated classes they can't be found.
How I can link them up correctly ? Do I have to move the generated Sources to the maven src/main/java section ? Do they need a package ? If yes, how can I set the package for the sources in the maven plugin ?
Here is My pom.xml form the antlr generation module:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>net-site-arti</artifactId>
<groupId>net.site.reverse</groupId>
<version>1.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>net-site-arti-antlr</artifactId>
<dependencies>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr4-runtime</artifactId>
<version>4.7.1</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.antlr</groupId>
<artifactId>antlr4-maven-plugin</artifactId>
<version>4.7.1</version>
<configuration>
<outputDirectory>src/main/generated-sources</outputDirectory>
</configuration>
<executions>
<execution>
<goals>
<goal>antlr4</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>src/main/generated-sources</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
And here is the pom.xml form the module where I want to implement the classes:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>net-site-arti</artifactId>
<groupId>net.site.reverse</groupId>
<version>1.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>net-site-arti-antlr-impl</artifactId>
<dependencies>
<dependency>
<groupId>net.site.reverse</groupId>
<artifactId>net-site-arti-antlr</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr4-runtime</artifactId>
<version>4.7.1</version>
</dependency>
</dependencies>
</project>
EDIT: Add build-helper-maven-plugin same result - the classes are in the root folder from the generated jar but the module that will use them don't pick up the classes.
Tried also with the maven-shade-plugin in this configuration
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern></pattern>
<shadedPattern>org.shaded.plexus.util.</shadedPattern>
<excludes>
<exclude>org.codehaus.plexus.util.xml.Xpp3Dom</exclude>
<exclude>org.codehaus.plexus.util.xml.pull.*</exclude>
</excludes>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
different jar but some result - in the jar the classes are under: org/shaded/plexus/util/ but the module can't import the classes.
Not a perfect solution but a solution:
In this post they say you can add the package name in a #header tag in the *.g-File. Unfortunately ANTLR doesn't generate the package in the Sourcefolder it only generated the Class-Files with the package declaration. So you have to generate them in the package:
here is an example pom:
<plugin>
<groupId>org.antlr</groupId>
<artifactId>antlr4-maven-plugin</artifactId>
<version>4.7.1</version>
<configuration>
<outputDirectory>src/main/java/net/site/antlr</outputDirectory>
</configuration>
<executions>
<execution>
<goals>
<goal>antlr4</goal>
</goals>
</execution>
</executions>
</plugin>
And if you add this to the g-File:
#header {
package net.site.antlr;
}
But you can't generate, with this solution, different grammar in different packages.
Attempting to get one jacoco report that will show all the results from multiple modules.
I am able to see that each of the sub-modules have a jacoco.exec after building the project but unsure of how to get it to output one report that will have all the results from every module combined.
This is what I have included in my Root pom.xml:
<plugin>
<groupId>#project.groupId#</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>#project.version#</version>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
</executions>
</plugin>
I created a new module explicitly for reporting purposes. (e.g. report-aggregate-module)
Deleted the group ids and used generic artifact ids for this example:
This is what I put in the pom.xml for this report-aggregate sub-module:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>report-aggregate</artifactId>
<groupId>com.name.group</groupId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<artifactId>report-aggregate</artifactId>
<name>Report Aggregate</name>
<properties>
<wildfly.version>10.0.0.Final</wildfly.version>
<wildfly.artifactId>wildfly-dist</wildfly.artifactId>
</properties>
<dependencies>
<dependency>
<groupId></groupId>
<artifactId>sub-module1</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId></groupId>
<artifactId>sub-module2</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId></groupId>
<artifactId>sub-module3</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId></groupId>
<artifactId>sub-module4</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>report-aggregate</id>
<phase>verify</phase>
<goals>
<goal>report-aggregate</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Everything seems to compile okay, the jacoco exec doesn't seem to get created for the report-aggregate-module. Anyone know a solution to this or if I'm doing this incorrectly?
Sorry if this comes a late answer.
I assume that your root pom.xml is an aggregater, with <modules> consisting of module1, module2, and report-aggregate. This is the cause of your trouble: as your root pom.xml is an aggregator, it runs JaCoCo BEFORE your submodules do, so your final report is empty. You should:
Move the configuration of goal report-aggregate of the jacoco-maven-plugin from the root POM to the report-aggregate POM. This should do the trick, because your report-aggregate POM uses <dependencies>, not <modules>.
Keep the configuration of goal prepare-agent of the jacoco-maven-plugin in the root POM.
I suggest you look at the JaCoCo forum https://groups.google.com/forum/#!topic/jacoco/FpdLbxsXSTY. It refers to a complete demo integration test https://github.com/jacoco/jacoco/tree/master/jacoco-maven-plugin.test/it/it-report-aggregate.
I have a maven project in which I'm trying to get a jar from a remote repository and add it to my local repository. After adding instead of referring its class, I need to run that jar which acts as a input for my current project.
I tried, and there are no build issues, but still it is not triggering to run the dependency.
My pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>secmaven</groupId>
<artifactId>secmaven</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>secmaven</name>
<description>secmaven</description>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.3</version>
<executions>
<execution>
<goals>
<goal>properties</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2</version>
<executions>
<execution>
<id>exec-one</id>
<phase>verify</phase>
<configuration>
<includeProjectDependencies>false</includeProjectDependencies>
<includePluginDependencies>true</includePluginDependencies>
<executableDependency>
<groupId>parent</groupId>
<artifactId>parent</artifactId>
</executableDependency>
<!-- Look up the main class from the manifest inside your dependency's JAR -->
<mainClass>mainclass</mainClass>
<arguments>
</arguments>
</configuration>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>parent</groupId>
<artifactId>parent</artifactId>
<version>0.0.1-SNAPSHOT</version>
<scope>system</scope>
<systemPath>C:\....\parent.jar</systemPath>
</dependency>
</dependencies>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
You can use a maven aggregator project pom for that.
Example:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.codehaus.mojo</groupId>
<artifactId>my-parent</artifactId>
<version>2.0</version>
<packaging>pom</packaging>
<modules>
<module>proj1</module>
<module>proj2</module>
</modules>
</project>
This way proj1 will always run before proj2.
I think is this what you want.
See https://maven.apache.org/pom.html#Aggregation for more information.
My project has dependencies lib-A and third-party lib-B:1.0 in my pom. But lib-A depends on lib-b:2.0. From my understanding, if lib-A had a shaded version of lib-b then that would solve the problem, correct? But the issue is lib-b is a third-party dependency which I have no control over.
Is there a work around so my project and lib-A will work correctly with different version of lib-b?
Workaround is to shade lib-b with your project.
Edit :
Create new project say shaded-lib-b with lib-b as dependency and in your project you need have dependency for shaded-lib-b and now package name of lib-b will be my.shaded.example
pom.xml for shaded-lib-b
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>my.shaded.example</groupId>
<artifactId>shaded-lib-b</artifactId>
<version>1.0</version>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>com.example</groupId>
<artifactId>lib-b</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>com.example</pattern>
<shadedPattern>my.shaded.example</shadedPattern>
</relocation>
</relocations>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer" />
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Within my project I have a sub-project auto-updater. Basically a jar file that is extracted and run when an update is available.
Is it possible to compile the sub-project, then place the outputted jar as a generated-resource so that the updater.jar is included in the final jar such as:
Project-1.0.jar
|-updater.jar
|-Main.class
|-B.class
Thanks in advance for any help(I'm new to Maven)
This task is calling for maven-assembly-plgin or maven-dependency-plugin
(I expect that updater is also maven project) this shoudl be proper configuration for maven-dependency-plugin [I did not test this, you might also need to put updater to project depndencies]
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.7</version>
<executions>
<execution>
<id>copy</id>
<phase>prepare-package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>company.com.project</groupId>
<artifactId>Updater</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>jar</type>
<outputDirectory>${project.build.outputDirectory}/classes</outputDirectory>
<destFileName>updater.jar</destFileName>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
It would be nice if your little jar is also built with Maven.
Then let's say your little jar POM's contains this:
<groupId>company.com.project</groupId>
<artifactId>Updater</artifactId>
<version>0.0.1-SNAPSHOT</version>
Then your Project-1.0 should use this dependency:
<dependency>
<groupId>company.com.project</groupId>
<artifactId>Updater</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
Add the sub project to your parent project as follows
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>1.0.0</modelVersion>
<groupId>Project-1.0</groupId>
<artifactId>myproject</artifactId>
<packaging>pom</packaging>
<version>1.0-SNAPSHOT</version>
<name>Project-1.0</name>
<modules>
<module>../updater</module>
</modules>
...
</project>
Then in you updater project pom file make the following changes
<?xml version="1.0"?>
<project>
<parent>
<artifactId>myproject</artifactId>
<groupId>Project-1.0</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>Project-1.0</groupId>
<artifactId>updater</artifactId>
<version>1.0-SNAPSHOT</version>
...
</project>
When you compile your parent project, the child project will automatically get compiled.
Then to add the sub project jar in your jar, add the following plugin in parent pom.xml
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>process-resources</phase>
<configuration>
<tasks>
<copy todir="${project.build.directory}/lib">
<fileset dir="${location of updater1.0.jar}"/>
</copy>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>