I have recently come across some libraries which provide their jar files with their dependencies bundled with them (sometimes referred to as an uber jar, fat jar, or shaded jar thanks to Maven's Shade plugin).
The problem is that I have two libraries (A and B) which contain
different versions of the same class file (same.fully.qulaified.class), and one of the libraries doesn't play nicely with the class file version in the other. I know I could play around with a custom classloader to sandbox the classes from A and B, but I'd rather go with a simpler, more standard solution if one exists.
Also, in general I'm surprised that these libraries are released as uber jars at all. It seems like when library providers package their dependencies in this way, this sort of problem is likely to show up.
So I have a couple questions:
1) Is it common and accepted practice for library providers to release uber jars?
2) Are there standard solutions to resolve conflicting transient dependencies (dependencies of dependencies) that show up at runtime when two of your required libraries have different versions of a class?
Related
I have two libraries as dependencies in my project. Each of those two libraries has bunch of libraries as dependency. However there are two same named libraries with different version, and one of them as I can see in maven pom dependency hierarchy is ommited conflicting with same named one. When I run program program it uses elements of both "parent" libs, but it uses one of conflicting "child" libs whit which it tries to work with "parent" lib which should use other conflicting lib. So is it possible to have both conflicting libs used in project.
Thanks
Not really.
You can use the Maven shade plugin to construct a shaded jar that contains the dependency with a different package hierarchy.
But the usual solution is to try to find a version of the library that works with all of the other stuff (not always possible, I know).
I am trying to add a dependency as jar file and that jar is shipped with all the classes which are needed for it to run.
compile files('lib/org.hl7.fhir.igpublisher.jar')
Along with that, I have few other dependencies which are added as maven dependencies.
compile ("ca.uhn.hapi.fhir:hapi-fhir-base:2.3")
Now I am facing a lot of issues related to class conflicts because same classes have been shipped with different versions.
In an ideal case, how should I solve this problem? I want to say that the local jar should always use its own files and other dependencies should ignore the local jar files.
Note:- I am using IntelliJ idea.
This is a tricky problem. There is only one classpath and multiple versions of the same class mean that only one of that versions is visible and the other ones are hidden.
One should generally avoid to declare dependencies on "fat jars" that contain their own dependencies. If possible, one should use the slim version without the dependencies (often both versions are published). If there is not alternative one can construct such a slim jar yourself by manually splitting up the jar file. It is also possible to control the structure by carefully ordering the dependencies on the classpath, but this is a little brittle.
I have a single very large codebase that compiles down to a JAR. I also use the shade plugin to compile it down and package it up with all dependencies. I also use the war plugin to get a WAR file.
With respect to the WAR file, once the goal is created I have a post-build event that simply copies the WAR file to its destination, so technically I'm good with that.
The problem I have is with the other two JARs. They both share the same artifactId since they're both built in a single pom.xml and this isn't acceptable for a number of reasons (including, but not limited to the fact that some caching of dependencies is pretty dumb about realizing that one JAR is the thin one and one is the full-dependency one).
What I need to do is create a pom.xml (or group thereof) suitable for builds (by Jenkins) that can use the same codebase (pulled from Github) but create two separate JAR files, each with their own artifactId.
Being a Maven novice, I've read through the beginning book and it seems to me that what I want is a parent pom.xml with two modules. But from what I can tell, each module means a separate directory with separate code. As I said, this is built from the same codebase. The only difference is one is built from the "regular" build, and the other is built using the "shade" plugin and goal.
The only other thing I can think of is build the "regular" JAR and then build the shaded JAR with a classifier of "full?" If this is the answer, may I humbly ask for some adult supervision on how to do this, as I'm not seeing how.
If that's not the answer, I suspect this must be a common problem, so again, some guidance would be very helpful!
The solution I came up with was to use a classifier for the "shaded" jar. Thus, the artifacts don't collide.
I then had an issue accessing it, but found the solution to that issue as well - How do I access a jar with a classifier?
I have a web app with thousands classes packed in hundred jar's placed in several folders.
I want create in local maven repository a library, containing all classes from those jars.
And I can use this dependensy in all my projects...
Like in a IntelliJ IDEA i create a global library. Select a folder with sets of jars with subfolders and set name "My web-app libs". And then i add this global lib on my project.
I have founded a way for construct group for several libraries, described here. But I have a hundred jar files and stupid make new project for each.
In advance thanks.
Look at maven shade plugin.
This plugin provides the capability to package the artifact in an uber-jar,
You may want to create another project which enlists all your artifacts as dependencies. So when you include it into your project all the needed 100 dependencies will be transitively resolved.
Maven Versions plugin could help you bulk updating the many versions inside this new artifact.
UPDATE
If all your 300+ jars are completely static, i.e. their versions are fixed, you might probably need to repack them with the shade plugin. Otherwise updating a version of a JAR from this huge set could be a trouble... I can't predict the performance, but my guess is that a normal Maven approach is more efficient.
A note on Maven shade plugin: you might need to move your shaded libraries to shaded package. That's what they usually do to avoid library versions conflicts. I.e. if your 300+ libs use spring-2.0 and your current project uses spring-3.1.0, both will be included anyway. So to avoid conflicts, it's recommended to configure this plugin to move spring-2.0 packages under a different package.
UPDATE 2
If your jars are not mavenized, Maven won't be a big help here. You should probably merge your jars manually and check if it works for you: Merging Multiple Jars in to a Single Jar.
I created an Eclipse 4 application and I needed a jar offering a functionality as part of my application (this could be anything e.g. log4j to make it trivial).
I added the jar as part of my project's classpath (Right Click->Configure Build Path) but on runtime my service failed with a ClassNotFound error (from OSGI I guess?).
Anyway searching this it turned out, at least as I have understand it, that I should add the jar as part of another Plugin and create a dependency from my application/service to this new plugin.
I.e. I created a Plugin Project from Existing JAR archives.
This time the setup worked.
So if I understand this, when developing for Eclipse/OSGi we should not add jars in the classpaths directly but add them via plugins (why?).
Question: If I am correct so far, what is the standard practice to include jars when developing a project?
Define/Create one Plugin Project from existing JAR archives and add all the required third party libraries needed there, or have a different plugin project per needed jar or something else perhaps???
Sorry if my terminology is not accurate. I am new in OSGi and Eclipse programming
Note: When talking about jars I am not refering to other OSGi services. I am refering to the norm of using ready, reliable third party libraries that would be needed by many parts of an application. E.g. log4j or an xml parsing library or apache commons etc
For the runtime it is always the Manifest and the headers there that control what is in your bundle classpath. There are three ways to get access to a jar:
Import-Package header. This is the recommended way. You define one import per package you need. You jar you want to access has to be deployed in the runtime as a bundle. It also needs to export all needed packages.
Require-Bundle . This is another way to access bundles. You define the id of the bundle you need and see all packages it exports. As Require-Bundle binds you more closely to the other bundle the Import-Package way should be preferred.
Bundle-Classpath . This allows to add jars to your classpath that you embed into your own bundle. This should only be a last resort when the other way do not work. You can have nasty classloading issues when mixing this with the other methods.
You can find many pre built bundles in maven central. Many jars today already contain an OSGi manifest. For the cases where this is not true many jars are repackaged as bundles by servicemix. See groupId: org.apache.servicemix.bundles. There is also the spring bundle repository where you find some more.
Below I listed some resources you might want to read:
http://www.aqute.biz/Blog/2007-02-19
http://wiki.osgi.org/wiki/Import-Package
http://wiki.osgi.org/wiki/Require-Bundle
http://www.vogella.com/blog/2009/03/27/required-bundle-import-package/
The examples you have mentioned are available as OSGi bundles, so you don't need to make them bundles yourself. You don't typically use direct jar dependencies in OSGi, you typically use package or bundle dependencies. In the log4j example you are referring to, you should use import package as there can be multiple bundle providers (newer log4j jar, springsource bundled version of older log4j, slf4j implementation...). This will disconnect your code dependencies from the actual provider.
These dependencies are maintained via you manifest, not your project classpath. In an eclipse plugin project, the projects build classpath is derived from the entries in the manifest.
Even though you are not using services, all code dependencies are still maintained via the manifest.
Extactaly same problem we faced in our project.
we have some legacy jar which are not OSGi compatible, we create lib folder parallel to BundleContent and added it into the classpath section of manifest.
Bundle-ClassPath: .,
/lib/<legacy jar>.jar
There is no need to exporting and importing of packages unnecessarily if only one bundle is going to consume it,