I have a Apache Commons library (commons-io.jar) added to my project classpath and I want to remove this library classes from my classpath during runtime and then I want to add after some work.
Is there any way to remove all classes from classpath during runtime and add these classes after some operation?
public static void main(String [] args){
//Some apache commons operations
//Remove apache commons classes from classpath
doSomeWork();
//Add apache commons classes to classpath
}
The problem you are experiencing is known as classpath hell.
There is no such functionality in Java. The reason behind this is the design of classloaders, which can only load classes and not unload them. In all likelyhood, this is not going to happen even in the future.
You could use a special classloader to load different classes from different jars, but even this way you quickly stumble into other limitations. For example, a classloader should only load the class if the parent has not yet loaded it. This means that other parts of your code cannot use apache-commons.jar. You could separate your program into two parts and have each run in its own classloader/classpath, but then you'll realize you can't share objects between them anymore. And so on.
In the end, you can either (a) try to find a combination of jars that works with both, (b) fix one part of the program so that it works with the other jar, or (c) break the program into two parts and have each run in separate JVMs.
You could use the maven-enforcer-plugin in your pom to force specific versions of the transitive dependencies.
Conflicting library version in a java maven project
Related
I have a Scala Akka Application which connects to HBase (currently CDP earlier HDP) deployed on rancher; Never faced any trouble when connecting to HDP hbase; Since recent HDP to CDP change, with the same image we are getting no method found on one of the dependency's class in one of the container, where as another container of same image connects to hbase properly.; even though the jar exists in the same image and classpath also.
one of the noticeable difference is change in the order of classpath.
Does change in the classpath order will effect the jars availability.
Does java libraries/classes would load in different order when they would hit a faster CPU cycle at startup.
What could be the reason for such "no class method found".
It certainly can, if the same class file is present in different classpath entries. For example, if your classpath is: java -cp a.jar:b.jar com.foo.App, and:
a.jar:
pkg/SomeClass.class
b.jar:
pkg/SomeClass.class
Then this can happen - usually because one of the jars on your classpath is an older version than the other, or the same but more complicated: one of the jars of your classpath contains a whole heap of different libraries all squished together and one of those components is an older version.
There are some basic hygiene rules to observe:
Don't squish jars together. If you have 500 deps, put 500 entries on your classpath. We have tools to manage this stuff, use them. Don't make striped jars, uber jars, etc.
Use dependency trackers to check if there are version difference in your dependency chain. If your app depends on, say, 'hibernate' and 'jersey', and they both depend on google's guava libraries, but hibernate imports v26 and jersey imports v29, that's problematic. Be aware of it and ensure that you explicitly decide which version ends up making it. Presumably, you'd want to explicitly pick v29 and perhaps check that hibernate also runs on v29*. If it doesn't, you have bigger problems. They are fixable (with modular classloaders), but not easily.
*) Neither hibernate nor jersey actually depend on guava, I'm just using them as hypothetical examples.
For example, if you use maven, check out the enforcer plugin. (groupId: org.apache.maven.plugins, artifactId: maven-enforcer-plugin).
My bet is that there is another version of the jar somewhere in CDP, and occasionally it is loaded before the version that you ship with your project, causing the error.
So, when your container starts, try logging from which location the conflicting class is loaded. This question might help you: Determine which JAR file a class is from
I am using different libraries that reference the same library in different versions with different group-ids (relocated). So maven assumes that those are different libraries and puts them both in classpath.
Nevertheless the package-names have not changed, so the same class with the same package-name exist in maven.
This is e.g. the case with org.bouncycastle:bcmail-jdk14-1.38 vs. org.bouncycastle:bcmail-jdk14-138. But since both versions are identical I don't expect trouble.
But with javax.xml.stream:stax-api-1.0-2 vs. stax:stax-api-1.0.1 we have no identical bytecode, so the same class with different content will be in the classpath.
Am I right - and if so, can maven warn me and prevent such problems?
You can use the enforcer rule
https://www.mojohaus.org/extra-enforcer-rules/banDuplicateClasses.html
that allows you to break the build if a class name appears in more than one jar.
No, for Maven those are different artifacts and you can get into trouble. Having multiple versions of the same class on the class path is not a good idea.
I think I already know the answer, but would appreciate if someone can confirm it for me.
A) We are including a library in a build using maven-shade-plugin;
B) This library implements singleton(s);
C) Our resulting jar (uber-jar) is, in turn, used in a build of a megaproject;
D) Other jars in a megaproject also using the same library (A);
Would the library (A) be not acting as a singleton across the entire megaproject?
Would we end up with a separate set of static variables for each shaded copy of an artifact?
Would using maven-assembly-plugin instead of maven-shade plugin help?
You described a scenario in which the same class could end up in the classpath more than once.
In such an application the class loader will pick up the first one found when looking for that class. Once it has found it, this class is loaded and initialized by this class loader, and he will not look it up again. Usually that does not lead to problems, and you will still have the singleton as you want: one instance only.
If - for some reason - you have multiple class loaders at hand, they each could load this class from another location. So you could end with several singleton instances.
The shade plugin seems not to be the best tool for that. I would suggest to use it only for standalone applications that you package in a single shaded JAR - the end product.
We always use the assembly plugin because it gives a more fine-grained control over the packaged assembly. But you should not use shaded JARs as dependencies, instead simply use the core libraries. Even if you have the same dependency in various dependency paths in your project, the assembly plugin will package it only once.
Assuming there are two jars of different library versions on a classpath, e.g.
java -cp A-2.1.jar:A-2.2.jar ...
The package and class names in the first and second jars are the same, but class implementation is different. Is it specified whether root jvm classloader will try to find a class in A-2.1 before A-2.2?
The problem is that AWS EMR adds hadoop jars to a classpath and some of its dependencies are of older versions. However, our application needs to use new versions of the same libraries, so will prepending the classpath with newer versions of libraries be enough or is shading a recommended practice in this case? http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-hadoop-config_hadoop-user-env.sh.html
From the Setting the Class Path documentation:
The order in which you specify multiple class path entries is
important. The Java interpreter will look for classes in the
directories in the order they appear in the class path variable.
That said, overriding the dependency JARs of another library will always be risky since the library provider might not have tested that combination, so you'll either need to ask them for reassurance, do your own testing, or shade/repackage the classes as you suggested.
an application I have written uses several third party jars. Sometimes only a small portion of the entire 50kB to 1.7mB jar is used - one or two function calls or classes.
What is the best way to reduce the jar sizes. Should I download the sources and build a jar with just the classes I need? What existing tools can help automate this (ex I briefly looked at http://code.google.com/p/jarjar/)?
Thank you
Edit 1:
I would like to lower the size of my third party 'official' jars like swingx-1.6.jar (1.4 MB), set-3.6 (1.7 MB) glazedlists-1.8.jar (820kB) , etc. so that they only contain the bare minimum classes I need
Edit 2:
Minimizing a jar by hand or by using a program like proguard is further complicated if the library uses reflection.
Injection with google guice does not work anymore after obfuscation with proguard
The answer by cletus on another post is very good How to determine which classes are used by a Java program?
Proguard would be an option. It can eliminate unused classes and methods. You can also use it to obfuscate, which can further reduce the size of your final jar. Be aware that class loading by name is liable to break unless care is taken to keep the affected classes unobfuscated.
I've found Proguard quite effective - can be a bit cryptic to understand at the outset. But I don't have any experience with similar to offer a comparison.
First of all, if you use only one class from JAR file this does not mean that this class does not use other classed from that JAR.
The option for you, if you use open source JARs, is to get sources of that JAR, attach them to your project, remove unnecessary stuff and build the changes by yourself.
You could add GenJar as an Ant task and use it to build the JAR. As it says on the library's home page,
GenJar is a specialized Ant task that
builds jar files based on class
dependencies rather than simply the
contents of a directory.
You can find it on SourceForge.