We Keep Coding

Method not found error in Castor-XML jar file

I have a simple maven project to do some SIF calls with MDM hub, and adding castor dependencies for this.
Maven dependency added:
org.codehaus.castor
castor-xml
1.4.1
This downloaded the castor-xml-1.4.1.jar file.
Right at the line calling sipClient.process(req) below exception is thrown
Exception in thread "main" java.lang.NoSuchMethodError: org.exolab.castor.xml.Marshaller.getResolver()Lorg/exolab/castor/xml/ClassDescriptorResolver;
at com.siperian.sif.message.CastorUtil.setMappingLoader(CastorUtil.java:470)
at com.siperian.sif.message.CastorUtil.beanToXmlString(CastorUtil.java:358)
at com.siperian.sif.message.CastorUtil.beanToXmlString(CastorUtil.java:323)
at com.siperian.sif.message.CastorUtil.beanToXmlString(CastorUtil.java:309)
at com.siperian.sif.message.CastorUtil.beanToXmlString(CastorUtil.java:295)
at com.siperian.sif.client.HttpSiperianClient._process(HttpSiperianClient.java:117)
at com.siperian.sif.client.SiperianClient.process(SiperianClient.java:179)
I can see the getResolver method and classDescriptorResolver in the jar file in Java Decompiler, Images
classResolverDescriptor
getResolver method
Same exception even for 1.3.2 dependency.
Should I download any extra dependencies.
Thanks

This specific error for 2 reasons:
1- You are missing the jar file that has this method (This might not be the issue you have, as you stated you can see it when looking at the decompiled jar)
2- You have 2 or more jars in your dependencies, and it is actually looking at the jar which does not have the method you need.
How you should approach this is as follows:
Go to your ide, and open your pom.xml file
Open the Dependency Heirarchy view and search for org.codehaus.castor or castor-xml and see how many different versions you have.
If you have more than 1, and some are included as part of another jar, you can use in your pom.xml to remove the versions which you dont want.
If you like command line you can probably do the above, using mvn dependency:tree
Hope this helps you in some way.
-- Edited --
Your code is using 1.3.2 dependency. How? You can download the castor-xml.1.3.2.jar and extract it and look into the Marshaller. You will see the method getResolver() does not take any parameters, and therefore you get NoMethodFound.
´´´
/**
* Returns the ClassDescriptorResolver for use during marshalling
*
* #return the ClassDescriptorResolver
* #see #setResolver
*/
public XMLClassDescriptorResolver getResolver() {
}
´´´
Therefore you need to find out in your dependency hierarchy, will one includes this 1.3.2 jar and exclude this jar from it.
An example of how to do exclude is in pom.xml:
<dependency>
<groupId>sample.group.which.has.castor.in.it</groupId>
<artifactId>artifactor.which.has.castor.in.it</artifactId>
<version>1.0</version>
<scope>compile</scope>
<exclusions>
<exclusion> <!-- declare the exclusion here -->
<groupId>org.codehaus.castor</groupId>
<artifactId>castor-xml</artifactId>
<version>1.3.2</version>
</exclusion>
</exclusions>
</dependency>
<dependency> <!-- add proper dependency also, as it is needed -->
<groupId>org.codehaus.castor</groupId>
<artifactId>castor-xml</artifactId>
<version>1.4.1</version>
</dependency>

How do I find Java dependencies?

I am following this tutorial. The code I have is this:
package com.ipponusa;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.streaming.Duration;
import org.apache.spark.streaming.api.java.JavaStreamingContext;
public class SparkStringConsumer {
public static void main(String[] args) {
SparkConf conf = new SparkConf()
.setAppName("kafka-sandbox")
.setMaster("local[*]");
JavaSparkContext sc = new JavaSparkContext(conf);
JavaStreamingContext ssc = new JavaStreamingContext(sc, new Duration(2000));
// TODO: processing pipeline
ssc.start();
ssc.awaitTermination();
}
}
My Maven dependencies are:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka_2.10</artifactId>
<version>1.6.0</version>
</dependency>
The problem is that the code and imports all have errors. I don't know what dependencies I need, and I don't know how to find out what dependencies I need. I am new to Java.
So my question is, how am I supposed to know what dependencies I need?
Note: Neither IntelliJ or Netbeans "Find Maven Dependency" features found anything for me.
Edit: If I go here I can find the Classes I am looking for, but I can't find the dependencies there.
Edit: If I go here I can find a long list of dependencies, but I can't find the ones that have the classes I need.
Thanks in advance!

I think you should add the spark core too
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.10</artifactId>
<version>2.1.0</version>
</dependency>

You can use the 'search by classname' feature of search.maven.org to find the library you need to import.
Also keep in mind that maven dependencies are transitive, i.e. a dependency of a dependency is included in your project too.

Not really Maven-specific, but you can look at the Spark source repo, search for the class you want, then look roughly at the folder that it is in.
For example, SparkContext exists under the core/ folder.
Note: spark-streaming-kafka depends on spark-streaming, which depends on spark-core, so if you included that, or the other core libraries like MLlib, SparkSQL, or GraphX, then core should be brought in as well.
Perhaps you needed to run mvn compile

Which class is loaded in a Maven project when multiple versions of same library are declared as dependencies?

The following is clearly bad practice but my question is out of pure curiosity: If I have a Maven project with two versions of the same library in the dependencies (e.g. jetty server in the pom.xml fraction below), then I import a class in my code (e.g. java code below), which version is the class picked up from?
<project>
....
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
<version>9.3.0.v20150612</version>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
<version>8.1.17.v20150415</version>
</dependency>
....
</project>
And the java code:
package test.hello;
import org.eclipse.jetty.server.Server;
public class Hello {
Server server;
public Hello() {
server = new Server();
}
}
Is the server object created from the old version class or the new version class? Maven does not complain about any ambiguity. It just loads a class; I don't know which.

Maven has nothing to do with loading jars or classes. Maven only gathers all the dependancy. Its upto your classloader to decide what to load when.
If you have two classes with the same binary name, and you want to know which one of them you are loading, you can only inspect the way that classloader tries to resolve a class name.

Why is Maven not resolving all dependencies for commons-configuration?

Summary
When trying XMLConfiguration configuration = new XMLConfiguration("config/config.xml"); with only commons-configuration 1.10 I need to add more depencies (namely commons-collections not newer than 3.2.1) to my maven setup. Why is that so and why doesn't maven simply resolve all needed dependencies?
Details
I am trying to get commons-configuration to work. First I wanted to use the latest version, 2.0-alpha2, which didn't work well at all since I was unable to configure Maven to download the correct ressources - but that is another story.
After I found out that version 1.10 is in fact "one point ten" (not "one point one zero") and thus the latest version of commons-configuration 1 (and covered by the tutorials), I decided to give it a try instead.
For my maven dependencies (integrated in eclipse) I used:
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.10</version>
</dependency>
However, when trying out this example:
package main;
import java.util.Iterator;
import org.apache.commons.configuration.ConfigurationException;
import org.apache.commons.configuration.XMLConfiguration;
public class ConfigurationTest {
public static void main(String... args) {
try {
XMLConfiguration configuration =
new XMLConfiguration("config/config.xml");
Iterator<String> iterator = configuration.getKeys();
while (iterator.hasNext()) {
System.out.println(iterator.next());
}
} catch (ConfigurationException e) {
e.printStackTrace();
}
}
}
with the following config.xml:
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<configuration>
<property>value</property>
<nestedproperty>
<arrayvalue>0,1,2,3,4</arrayvalue>
<property>anothervalue</property>
</nestedproperty>
</configuration>
I got the error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/collections/CollectionUtils
at org.apache.commons.configuration.XMLConfiguration.constructHierarchy(XMLConfiguration.java:640)
at org.apache.commons.configuration.XMLConfiguration.initProperties(XMLConfiguration.java:596)
at org.apache.commons.configuration.XMLConfiguration.load(XMLConfiguration.java:1009)
at org.apache.commons.configuration.XMLConfiguration.load(XMLConfiguration.java:972)
at org.apache.commons.configuration.XMLConfiguration$XMLFileConfigurationDelegate.load(XMLConfiguration.java:1647)
at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:324)
at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:261)
at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:238)
at org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:184)
at org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.<init>(AbstractHierarchicalFileConfiguration.java:95)
at org.apache.commons.configuration.XMLConfiguration.<init>(XMLConfiguration.java:261)
at main.ConfigurationTest.main(ConfigurationTest.java:12)
I first hoped they (not me, of course) just screwed up some maven dependencies and since I wouldn't bother which version to use anyway anymore (I didn't get 2.0 to work, remember?) I decided to go down to version 1.9 by replacing the maven dependency with:
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.9</version>
</dependency>
That solved the problem pretty well, the test case is running:
property
nestedproperty.arrayvalue
nestedproperty.property
But when I tried to implement a similar example to the one referenced in Very simple Apache-commons configuration example throws NoClassDefFoundError and its follow-up question I got the exact same error which is referenced there - but the solution, importing org.apache.commons.beanutils.PropertyUtils is not working as I am missing the beanutils. So basically by downgrading I just switched from the error of missing the collections to missing beanutils.
There is a dependency overview where you can see which dependencies are used when you do what. I was a bit suprised to learn that version 1.10 now used other dependencies (namely the CollectionUtils) than 1.9 did in the constructor call. Since there were dependency problems in 1.10 as well as in 1.9 I just sticked to the newer version.
I found the CollectionUtils located in the following artifact (as I was pointed there by its maven repository):
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>4.0</version>
</dependency>
Sadly that one (not obvious to me at first) doesn't define the class CollectionUtils in the package collections, but in the package collections4. It was hinted at this problem on the dependency overview, but they only mentioned possible problems with earlier versions... I appeared to be at a point of not thinking much about it anymore but simply changed the dependency to:
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
<version>3.2.1</version>
</dependency>
I got everything to work (more or less, but the Exceptions I get now are not anymore depending on missing class definitions) after using these dependencies:
<dependencies>
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.10</version>
</dependency>
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
<version>3.2.1</version>
</dependency>
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
<version>1.9.2</version>
</dependency>
</dependencies>
Why do I have to add the dependencies myself? I thought the whole point in using maven is to avoid having to do such things and in terms of javadocs and source files it does a pretty good job.
By now I am convinced that the dependencies are not included in the hierarchy by design (is that so?), probably to avoid overhead. However is there a way to either simply get all dependencies at once or even better to get all dependencies I need? And why is it designed this way?

If we analyse commons-configuration's POM we see that the commons-collections dependency is optional:
<dependencies>
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
<version>3.2.1</version>
<optional>true</optional>
</dependency>
...
Furthermore, from the Maven docs:
If a user wants to use functionality related to an optional
dependency, they will have to redeclare that optional dependency in
their own project.

This issue is explained on the Runtime dependencies page of the Commons Configuration website.
Quoting from that page:
A lot of dependencies are declared in the Maven POM. These are all needed during compile time. On runtime however you only need to add the dependencies to your classpath that are required by the parts of the Commons Configuration package you are using. The following table helps you to determine which dependencies you have to include based on the components you intend to use.
The other answers explain why this works from a Maven perspective. This answer is intended to provide a defence, of sorts, to the Commons Configuration folks. They did at least warn you!
In cases where the dependencies are on other Apache Commons components, they've taken the time to test with a variety of versions and have posted information on compatibility at the bottom of that page.

Maven tries to resolve all necessary dependencies for a library you're using in your pom. Well sometimes you have some dependencies which are only necessary for some specific features and you don't want to force the user of your dependency to download it if he doesn't use it. Then you're declaring your dependency as optional. This happened with commons-collections within commons-configuration. See commons-configuration-pom here

WebSphere MQ Configuration issues

getting below error after i configure MQ connection factory.
java.lang.ClassCastException: com.ibm.ejs.jms.JMSQueueConnectionFactoryHandle incompatible with com.ibm.mq.jms.MQQueueConnectionFactory
my code snippet where the exception is pointing to :
String queueConnectionJndi = props.getProperty(queueConnection + MQ_CONN);
queueConnectionFactory = MQQueueConnectionFactory)initialContext.lookup(queueConnectionJndi);
I am not able to find out the root cause of this.
can any body please help me on this, Thanks in advance.

There is no way to be sure without more context, but it looks like this method call:
initialContext.lookup(queueConnectionJndi);
is returning an object of type com.ibm.ejs.jms.JMSQueueConnectionFactoryHandle which cannot be cast to com.ibm.mq.jms.MQQueueConnectionFactory.
Can you provide more context?

This Post on old nabble sounds like a similar issue and may help you out.
Specifically the final response talks about removing any jms.jar file(s) that may be in your deployed WAR. Check your WEB-INF/lib. Certain jars are provided by the Websphre container and shouldn't be including them in your WAR.
This Post on the spring fourm also indicates issues of this nature caused by jars included in the classpath that shouldn't be there
Remove any of the following if you find them...
naming.jar
providerutil.jar
jndi.jar
jms.jar
mq.jar
websphere.jar

Can you rewrite your code to use JMS standard (ConnectionFactory or QueueConnectionFactory)instead of a Websphere MQ specific implementation class? That way you won't be tying your app to Websphere MQ and porting it to an alternative MQ implementation would be easier...
i.e.
import javax.jms.QueueConnectionFactory;
...
queueConnectionFactory = (QueueConnectionFactory)initialContext.lookup(jndiName);

the MQ jars which WAS is using and my application using are different so this problem occured. when i corrected the classpath it is resolved. sorry for the trouble, thanks for the help.

I went through a lot of trial and error to find the answer (the answer to my question at least). I hope this solution will solve your issues too. As mentioned from another post excluding the jms library works. But how do you exclude the jms library and still be able to compile the code? That was something no one seems to have mentioned. The solution to that is to make the scope for the jms library to "provided" (if you are using Maven or Gradle).
As mentioned somewhere:
"Provided means that you need the JAR for compiling, but at run time there is already a JAR provided by the environment so you don't need it packaged with your app. For a web app, this means that the JAR file will not be placed into the WEB-INF/lib directory."
So in your pom.xml add/update these:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jms</artifactId>
<version>4.3.4.RELEASE</version>
<exclusions>
<exclusion>
<groupId>javax.jms</groupId>
<artifactId>javax.jms-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>javax.jms</groupId>
<artifactId>jms</artifactId>
<version>1.1</version>
<scope>provided</scope>
</dependency>
Hopefully this can be helpful to those who have been frustrated by the lack of answers from the Internet.
Remove all the ibm libraries. They are useless. Once you deploy onto Websphere, it will use its libraries anyways.