Build lucene from sources: Codec does not exist - java

I've imported Lucene sources and built successfully.
But when I'm trying to use any of Lucene classes, I get
A SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene410' does not exist
The current classpath supports the following names: []
I tried to get path to classes by
String path = Lucene410Codec.class.getProtectionDomain().getCodeSource().getLocation().toURI().getPath();
And got right path, so there is no problem with wrong jar-file.

The problem was that I've missed META-INF folder while imported project.
I've manually added META-INF/services folder and it's contents - codecs files (which I took from lucene.core.jar) to sources and configured right build path.
add something to resources in eclipse
Now I can work with Lucene.

For the ones who is facing such issue I have another solution which seems decent. Main reason for such issue to occur is that some JAR files (Lucene in this case) providing implementations of some interfaces ship with 'META-DATA/service' directory. This directory then maps interfaces to their implementation classes for lookup by the service locator. So the solution will be to relocate the class names of these implementation classes and merge multiple implementations of the same interface into one service entry.
Maven's shade plugin provides a resource transformer called ServiceResourceTransformer which does such relocation. So in practice I would define the plugin as follows:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>main.class.of.your.app.MainClass</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
</configuration>
</execution>
</executions>
</plugin>

Related

Does maven shade plugin automatically downcase the package names on Windows?

I have two local artifacts: one with com.org.abc, another with COM.org.xyz. I have created a shaded jar including these 2 and all the other needed dependencies.
WHEN I CREATE A SHADED JAR ON LINUX, 2 SEPARATE FOLDERS ARE CREATED : com and COM. BUT ON WINDOWS ONLY SINGLE FOLDER IS CREATED.
When I create a shaded jar on windows, it creates a single folder: com.org with folders abc and xyz inside. No separate uppercase COM folder is created. Therefore the code dependent on uppercase COM package fails with could not initialize class error.
(I didn't name the above 2, they were created and distributed individually by 2 separate teams and many teams have been using these jars so changing the package name is a long cycle)
Maven config:
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
<executions>
<execution>
<id>add-mylocal</id>
<phase>clean</phase>
<configuration>
<file>${jars.path}/mylocal.jar</file>
<repositoryLayout>default</repositoryLayout>
<groupId>com.org</groupId>
<artifactId>mylocal</artifactId>
<version>1.0</version>
<packaging>jar</packaging>
<generatePom>true</generatePom>
<localRepositoryPath>${local.repo.path}</localRepositoryPath>
</configuration>
<goals>
<goal>install-file</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<source>1.7</source>
<target>1.7</target>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<manifestEntries>
<Build-Version>${buildversion} (${timestamp})</Build-Version>
</manifestEntries>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
Any solution to make it work on windows?
Documenting the discussion from the comments as an answer for posterity:
The issue here isn't maven-shade-plugin, which doesn't modify (or even care about) the case of the package. The issue here is that the underlying [windows] filesystem is case-insensitive, and does not differentiate between com and COM.
Unless you're willing to change the package names, there's no workaround from within maven-shade-plugin. You'll have to use a case-sensitive file system. Once appealing option would be to use the Windows Subsystem for Linux, which provides its own case-sensitive file system (ext4, IIRC).
There's also a method to make an NTFS filesystem case sensitive (see, e.g., this SU thread), but I've never done so myself, and can't recommend it based on personal experience.
While maven shade actually seems to delete the uppercase COM directory, it actually merges it with the lowercase one.
This distinction seems small, but allows us to use a special part of maven shade to solve this, namely, the relocation feature.
Using this feature, we can relocate the weirdly named uppercase library to lowercase, without requiring any changes in the source of those libraries.
<relocations>
<relocation>
<pattern>COM.org.xyz</pattern>
<shadedPattern>com.org.xyz</shadedPattern>
</relocation>
</relocations>
Your final shade config will look like:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<source>1.7</source>
<target>1.7</target>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<manifestEntries>
<Build-Version>${buildversion} (${timestamp})</Build-Version>
</manifestEntries>
</transformer>
</transformers>
<relocations>
<relocation>
<pattern>ME.ferrybig.uppercase.com</pattern>
<shadedPattern>me.ferrybig.uppercase.com</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
I found the solution to the above problem. Although the best practice, as also suggested by Ferrybig and Mureinik in their answers, is to enforce the lowercase package name standards to the projects but since in my case this was not possible, I have followed following approach.
Issue in short:
On Windows, shade plugin was merging COM folders with com , because Windows treat them as case-insensitive so if a package com is already created, it would add contents of COM in this only rather than creating new one.
Solution:
In my shade plugin, I have created 2 uber jars - one containing uppercase COM packages and second with all the other dependencies. This solved the issue because because there was no conflict with com in the first jar as it containing only COM.
The configuration that I used was from this post.
Basically in first execution block, I included artifacts containing COM packages and excluded the same from second execution block:
Execution block 1:
<include><artifact_name_with_COM_package></include>
Execution block 2:
<exclude><artifact_name_with_COM_package></exclude>
NOTE: Once again, the first choice should be to enforce naming standards in packages. But if you want a quick workaround you can try this.

Maven: exclude dependency from shade plugin

I've seen next string after mvn clean install
Including com.sun.jersey.contribs:jersey-multipart:jar:1.5 in the
shaded jar
Problem: I can't make it not shaded even I've added exlusion for maven-shade-plugin (see code below)
My maven-shade-plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<artifactSet>
<excludes>
//Here ==> <exclude>com.sun.jersey.contribs:jersey-multipart:jar</exclude>
</excludes>
</artifactSet>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<manifestEntries>
<Main-Class>Main</Main-Class>
<Build-Number>123</Build-Number>
</manifestEntries>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.handlers</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.schemas</resource>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
According to http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html, your exclusion syntax is wrong:
Artifacts to include/exclude from the final artifact. Artifacts are denoted by composite identifiers of the general form groupId:artifactId:type:classifier. ... For convenience, the syntax groupId is equivalent to groupId:*:*:*, groupId:artifactId is equivalent to groupId:artifactId:*:* and groupId:artifactId:classifier is equivalent to groupId:artifactId:*:classifier.
So either use com.sun.jersey.contribs:jersey-multipart:*:jar or com.sun.jersey.contribs:jersey-multipart for your exclusion.
<artifactSet>
<excludes>
<exclude>com.sun.jersey.contribs:jersey-multipart</exclude>
</excludes>
</artifactSet>
Add scope tag in dependency tag with value as 'provided'. It will exclude that dependency.
For anyone, who made a lot of steps to exclude some transitive artifact from your fat jar and it still presented, there may be two causes:
This dependency is transitive for other dependency, not that one, where you've made exclusion. Thus you need to add exclusion tag somewhere else (if your exclusion is located in <dependencies> section, not plugin-level).
mvn dependency:tree is very useful for these searches.
These dependency classes comes to your jar straight from some dependency artifact as ordinary classes, not transitive dependency.
There is some chance that one of dependencies of your project represents fat jar. Certain classes which you want to exclude comes to your project as ordinary classes, not transitive dependency. Thus Maven unable to exclude it. Also this is a reason that Maven’s provided scope is not working (classes still presented in your jar).
More details here: https://mchesnavsky.tech/maven-dependency-exclusion-not-working/

Using Different Properties Files for Runnable Jars, Maven Eclipse

I am new to Maven. I apologize if this is a silly question. I have a Maven Eclipse project with a pom file that creates a runnable jar using the shade plugin. My plan is to include one of several properties files in the build (please correct me if my terminology is incorrect. I really am green). I have multiple properties files located in src/main/resources.
I would like to find a way to include in the runnable jar only the properties file that applies. Specifically, I have two properties files (config1.properties, config2.properties) in my project and each build will only use one of these files depending on the desired functionality of the resulting jar. The properties file would be reassigned a generic name, specifically "defaultconfig.properties", so that it plays nicely with the code.
The resulting jar files (a different one for each different properties file) will run as separate cron jobs. I think I will be using different Jenkins projects to deploy the runnable jars to specific servers based on what task the properties file configures the project for (I think I got this part). The cron jobs will run on these servers.
my questions are:
1) where does the logic go that determines which properties file to include in the runnable jar?
for example, suppose I have config1.properties (that specifies to use var1 = blah and var2 = blip and var3 = boink) and config2.properties (which specifies to use var1 = duh var2 = dip and var3 = doink). Where do I tell Maven which of these two files to use for specific builds?
2) am I supposed to pass in a parameter that tells Maven which properties file to use? If so, who do I pass this parameter too? Is this something that can be configured in a Jenkins project? (even just some basic reading would help me here, as I am not sure if this would be a maven issue or a jenkins issue. I am green with both.)
Here is a the relevant portion of my pom.xml file:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.foo.dashboard.bar.Runner</mainClass>
</transformer>
</transformers>
<finalName>FooBar</finalName>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
I have tried using IncludeResourceTransformer to rename the properties file and but was not sure how include the logic for switching properties files based on desired jar file functionality.
Thank you.
This is what I came up with. I would love to know other approaches or the best way to do this however. Being totally new to all of this.
Create separate executions in the same plugin. Note: They must be assigned different <id> values.
Each <execution> can be configured differently. To rename the resources that you are including use the <transfomer>implementation = implementation="org.apache.maven.plugins.shade.resource.IncludeResourceTransformer">.
To exclude resources that are not needed (in my case, other properties files) use <transformer> implementation = "org.apache.maven.plugins.shade.resource.DontIncludeResourceTransformer">. This is more nicety than necessity. Why bundle up unneeded files in your jar?
Make sure each <execution> has a different <finalName> so that the resulting jars don't overwrite each other.
Here is my pom file.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<id>ID1</id>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.foo.dashboard.bar.Runner</mainClass>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.IncludeResourceTransformer">
<resource>defaultconfig.properties</resource>
<file>src/main/resources/defaultconfig_1.properties</file>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.DontIncludeResourceTransformer">
<resource>defaultconfig_2.properties</resource>
</transformer>
</transformers>
<finalName>FooBar1</finalName>
</configuration>
</execution>
<execution>
<id>ID2</id>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.foo.dashboard.bar.Runner</mainClass>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.IncludeResourceTransformer">
<resource>defaultconfig.properties</resource>
<file>src/main/resources/defaultconfig_2.properties</file>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.DontIncludeResourceTransformer">
<resource>defaultconfig_1.properties</resource>
</transformer>
</transformers>
<finalName>FooBar2</finalName>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
I know this is not the final answer since this set up yields multiple jar files and I was originally looking for a solution that would give me only the jar file associated with a specific properties file, not all of the jar files from all of the properties files.
So I'm still taking suggestions on how to improve this solution so that the build would only a yield a single jar.

Maven shade + resteasy Could find writer for content-type

I have a project that works fine with maven managed dependencies. But I have a requirement for giving my jar files as one.
For this I use maven-shade plugin (http://maven.apache.org/plugins/maven-shade-plugin/). All class files are exported correctly but when I try to run my application I get an error as:
Could find writer for content-type multipart/form-data type: org.jboss.reasteasy.plugins.provider.multipart.MultipartFormDataOutput
Any help would be great, thanks.
Note: I had a similar problem with spring whose main cause is configuration files. Many jar files contained a configuraiton file that has same name. All configuration files tries to override the others. After merging that file with maven-shade configuration problem was solved.
You maybe missing one of the Shade transformers listed below. I was seeing the same error as yours when running 'java -jar' on my Shade-built jar file. Make sure you have a org.apache.maven.plugins.shade.resource.ServicesResourceTransformer entry. JAR files providing implementations of some interfaces often ship with a META-INF/services/ directory that maps interfaces to their implementation classes for lookup by the service locator. To merge multiple implementations of the same interface into one service entry, the ServicesResourceTransformer can be used. I believe this was the case with RestEasy running under Shade.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>path.to.your.App</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.handlers</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.schemas</resource>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

configure Maven Shade minimizeJar to include class files

I am trying to minimize the UberJar's size by using Maven Shade Plugin's minimizeJar. It looks like minimizeJar only includes classes that are statically imported in the code (I suspect this because I see LogFactory.class in uber jar at org\apache\commons\logging\ but no classes of the impl package are included, hence throwing java.lang.ClassNotFoundException: org.apache.commons.logging.impl.LogFactoryImpl when I run the uber-jar).
Is there any way I can tell Maven's Shade plugin to include specified packages into the final jar no matter what the minimizeJar suggests?
Here the pom snippet of what I am trying:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>1.5</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<minimizeJar>true</minimizeJar>
<filters>
<filter>
<artifact>commons-logging:commons-logging</artifact>
<includes>
<include>org/apache/commons/logging/**</include>
</includes>
</filter>
</filters>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.myproject.Main</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
This functionality has been added to version 1.6 of the maven-shade-plugin (just released). minimizeJar will now not remove classes that have been specifically included with filters. Note that including some of an artifact's classes in a filter will exclude non-specified classes for that artifact, so be sure to include all the classes that you need.
Here's an example plugin config:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<minimizeJar>true</minimizeJar>
<filters>
<filter>
<artifact>log4j:log4j</artifact>
<includes>
<include>**</include>
</includes>
</filter>
<filter>
<artifact>commons-logging:commons-logging</artifact>
<includes>
<include>**</include>
</includes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
I am using the 2.0 version of the Maven Shade Plugin and still I am not able to include classes after "minimizing" the JAR.
As a workaround the only thing that comes to my mind is to create references to the needed classes to avoid the minimization code to get rid of them. ie:
/*
* This block prevents the Maven Shade plugin to remove the specified classes
*/
static {
#SuppressWarnings ("unused") Class<?>[] classes = new Class<?>[] {
JButton.class,
JMenu.class,
JMenuBar.class,
JMenuItem.class
};
}
I hope Maven developers implement a way to handle this situation (which I guess is very common).

Categories