I have Kafka 0.8 built using Scala 2.8; the steps followed from this site. I could also run the kafka-console-producer and kafka-console-consumer examples successfully. I am now experimenting with Consumer Group Example (Please the "Full Source Code" section). The exception I get is below:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
at java.lang.Thread.run(Thread.java:701)
Caused by: java.lang.NoClassDefFoundError: scala/Tuple2$mcJJ$sp
at kafka.consumer.ConsumerConfig.<init>(Unknown Source)
at ConsumerGroupExample.createConsumerConfig(ConsumerGroupExample.java:55)
at ConsumerGroupExample.<init>(ConsumerGroupExample.java:18)
at ConsumerGroupExample.main(ConsumerGroupExample.java:64)
... 6 more
Caused by: java.lang.ClassNotFoundException: scala.Tuple2$mcJJ$sp
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
... 10 more
My dependencies looks like this:
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
<version>0.8.0</version>
</dependency>
<dependency>
<groupId>javax.inject</groupId>
<artifactId>javax.inject</artifactId>
<version>1</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.8.0</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>com.101tec</groupId>
<artifactId>zkclient</artifactId>
<version>0.3</version>
</dependency>
<dependency>
<groupId>com.yammer.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>2.2.0</version>
</dependency>
As a note I could get it running with a Python client. But I really need this done in Java. Would really appreciate any help!
You're using kafka built against 2.10 (see, that _2.10 in artifact name)
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
<version>0.8.0</version>
</dependency>
with scala 2.8.0
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.8.0</version>
</dependency>
Major scala versions (2.8.*, 2.9.*, 2.10.*) are binary incompatible between each other -- pick the right version of either kafka artifact or project's scala library and everything will be fine.
Related
I am trying to run the mahout code in the Hadoop cluster, it is throwing the below error.
Error: java.lang.ClassNotFoundException:
org.apache.mahout.cf.taste.model.DataModel at
java.net.URLClassLoader.findClass(URLClassLoader.java:387) at
java.lang.ClassLoader.loadClass(ClassLoader.java:418) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at
java.lang.ClassLoader.loadClass(ClassLoader.java:351) at
java.lang.Class.forName0(Native Method) at
java.lang.Class.forName(Class.java:348) at
org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2625)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2590)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2686)
at
org.apache.hadoop.mapreduce.task.JobContextImpl.getReducerClass(JobContextImpl.java:211)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:614)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)
Although it's working fine in local. Here's my pom.xml
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>3.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.24</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.mahout</groupId>
<artifactId>mahout-core</artifactId>
<version>0.9</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-math3</artifactId>
<version>3.6.1</version>
</dependency>
</dependencies>
I'm currently trying to use apache POI to make excel files in java, SXSSF to be exact. When running it however, I'm getting the following error:
Caused By: java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.xmlbeans.impl.schema.SchemaTypeLoaderImpl.build(SchemaTypeLoaderImpl.java:161)
at org.apache.xmlbeans.impl.schema.SchemaTypeSystemImpl.<init>(SchemaTypeSystemImpl.java:198)
at org.apache.xmlbeans.metadata.system.sXMLSCHEMA.TypeSystemHolder.<init>(TypeSystemHolder.java:9)
Truncated. see log file for complete stacktrace
Caused By: org.apache.xmlbeans.XmlRuntimeException: java.lang.ClassCastException: org.apache.xmlbeans.impl.schema.SchemaTypeSystemImpl cannot be cast to org.apache.xmlbeans.SchemaTypeLoader
at org.apache.xmlbeans.impl.schema.SchemaTypeLoaderImpl.build(SchemaTypeLoaderImpl.java:164)
at org.apache.xmlbeans.impl.schema.SchemaTypeSystemImpl.<init>(SchemaTypeSystemImpl.java:198)
at org.apache.xmlbeans.metadata.system.sXMLTOOLS.TypeSystemHolder.<init>(TypeSystemHolder.java:9)
at org.apache.xmlbeans.metadata.system.sXMLTOOLS.TypeSystemHolder.<clinit>(TypeSystemHolder.java:6)
at java.lang.Class.forName0(Native Method)
Truncated. see log file for complete stacktrace
Caused By: java.lang.ClassCastException: org.apache.xmlbeans.impl.schema.SchemaTypeSystemImpl cannot be cast to org.apache.xmlbeans.SchemaTypeLoader
at org.apache.xmlbeans.impl.schema.SchemaTypeLoaderImpl.build(SchemaTypeLoaderImpl.java:162)
at org.apache.xmlbeans.impl.schema.SchemaTypeSystemImpl.<init>(SchemaTypeSystemImpl.java:198)
at org.apache.xmlbeans.metadata.system.sXMLTOOLS.TypeSystemHolder.<init>(TypeSystemHolder.java:9)
at org.apache.xmlbeans.metadata.system.sXMLTOOLS.TypeSystemHolder.<clinit>(TypeSystemHolder.java:6)
at java.lang.Class.forName0(Native Method)
Truncated. see log file for complete stacktrace
Now, I know the similar issues to this have come up, and the usual fix is to alter weblogic.xml, and make
<prefer-web-inf-classes>false</prefer-web-inf-classes>
<wls:container-descriptor>
<wls:prefer-application-packages>
<wls:package-name>org.apache.commons.collections4.*</wls:package-name>
<wls:package-name>org.apache.commons.compress.*</wls:package-name>
<wls:package-name>org.apache.poi.*</wls:package-name>
<wls:package-name>org.apache.xmlbeans.*</wls:package-name>
<wls:package-name>org.apache.xmlbeans.XmlOptions</wls:package-name>
<wls:package-name>org.openxmlformats.*</wls:package-name>
<wls:package-name>schemaorg_apache_xmlbeans.*</wls:package-name>
</wls:prefer-application-packages>
<wls:prefer-application-resources>
<wls:resource-name>schemaorg_apache_xmlbeans/system/sXMLCONFIG/TypeSystemHolder.class</wls:resource-name>
<wls:resource-name>schemaorg_apache_xmlbeans/system/sXMLLANG/TypeSystemHolder.class</wls:resource-name>
<wls:resource-name>schemaorg_apache_xmlbeans/system/sXMLSCHEMA/TypeSystemHolder.class</wls:resource-name>
<wls:resource-name>schemaorg_apache_xmlbeans/system/sXMLTOOLS/TypeSystemHolder.class</wls:resource-name>
</wls:prefer-application-resources>
</wls:container-descriptor>
However, the issue arises that setting prefer-web-inf-classes to false breaks some of the rest of the application. So my question becomes, is there a way in the file I'm creating the excel, to specify that I want it to be built from the apache xmlbeans and not the weblogic xmlbeans? I'm thinking something like Classloader/Urlloader but I haven't been able to make that work properly yet, but I may be implementing it wrong.
Lastly, I'll include my pom.xml file
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>${org.apache.poi-version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.16.0</version>
</dependency>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>${org.apache.commons-commons-codec-version}</version>
<exclusions>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>${org.apache.commons-commons-lang3-version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.persistence</groupId>
<artifactId>org.eclipse.persistence.core</artifactId>
<version>${org.eclipse.persistence-core-version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>2.10.14</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
<version>1.9.3</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-to-slf4j</artifactId>
<version>2.8.2</version>
</dependency>
</dependencies>
</project>
To be more specific, in my case I had added the following code to weblogic-application.xml for fixing the exception - java.lang.ClassCastException: org.apache.xmlbeans.impl.schema.SchemaTypeSystemImpl cannot be cast to org.apache.xmlbeans.SchemaTypeLoader
<weblogic-application>
<prefer-application-packages>
<package-name>org.apache.*</package-name>
<package-name>schemaorg_apache_xmlbeans.*</package-name>
</prefer-application-packages>
<prefer-application-resources>
<resource-name>schemaorg_apache_xmlbeans/*</resource-name>
</prefer-application-resources>
</weblogic-application>
I'm working on a Google Dataflow Java project. I'm trying to create and update BigQuery tables in the initialisation of the pipeline. So, for getting started, I'm trying to instantiate BigQuery Java Client this way:
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
When running, it brings me the following exception:
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/api/gax/retrying/ExceptionRetryAlgorithm
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at com.google.cloud.BaseService.<clinit>(BaseService.java:48)
at com.google.cloud.bigquery.BigQueryOptions$DefaultBigQueryFactory.create(BigQueryOptions.java:46)
at com.google.cloud.bigquery.BigQueryOptions$DefaultBigQueryFactory.create(BigQueryOptions.java:40)
at com.google.cloud.ServiceOptions.getService(ServiceOptions.java:426)
at com.mydomain.Analysis.main(Analysis.java:359)
Caused by: java.lang.ClassNotFoundException: com.google.api.gax.retrying.ExceptionRetryAlgorithm
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 17 more
This is my pom.xml:
<properties>
<beam.version>2.19.0</beam.version>
</properties>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.apache.commons/commons-collections4 -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>4.0</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bigquery</artifactId>
<version>1.110.0</version>
</dependency>
<!-- Beam -->
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-core</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
<version>${beam.version}</version>
<exclusions>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-extensions-google-cloud-platform-core</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-jdbc</artifactId>
<version>${beam.version}</version>
</dependency>
</dependencies>
What am I missing?
From the error it looks like some .jar containing the com.google.api.gax.retrying.ExceptionRetryAlgorithm.class is missing.
A quick Google search directed me to add the below dependency to the pom.xml. This gax.jar contains the ExceptionRetryAlgorithm class.
<dependency>
<groupId>com.google.api</groupId>
<artifactId>gax</artifactId>
<version>0.14.0</version>
</dependency>
I hope that adding the above dependency to your pom resolves the issue.
Here what i got when compile the project
by this command
mvn compile exec:java -Dexec.classPathScope=compile -Dexec.mainClass=trident.myproject
got this
java.lang.NoClassDefFoundError: storm/trident/state/StateFactory
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)
at java.lang.Class.getMethod0(Class.java:2670)
at java.lang.Class.getMethod(Class.java:1603)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:285)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.ClassNotFoundException: storm.trident.state.StateFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 6 more
INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] An exception occured while executing the Java class. storm/trident /state/StateFactory
storm.trident.state.StateFactory
i reviewed the folder and can't find this class ! although i ran it before successfully
i wrote pom file as parts
<dependencies>
<dependency>
<groupId>storm</groupId>
<artifactId>storm</artifactId>
<version>0.8.2</version>
<scope>provided</scope>
</dependency>
part 2 of pom.xml
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>colt</groupId>
<artifactId>colt</artifactId>
<version>1.2.0</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.0.0</version>
</dependency>
part 3 of pom.xml
<dependency>
<groupId>org.twitter4j</groupId>
<artifactId>twitter4j-core</artifactId>
<version>4.0.2</version>
</dependency>
<dependency>
<groupId>org.twitter4j</groupId>
<artifactId>twitter4j-stream</artifactId>
<version>4.0.2</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
</dependencies>
You might have changed the scope of the dependency in your pom.xml file. If your container is providing the dependency change scope to provided.
<scope>provided</scope>
If container is not providing the dependency change scope to compile.
<scope>compile</scope>
I am using this doc.
But pom.xml is bit changed (because dependencies from example is not resolved and i have maven error messages), so part of pom.xml i next:
<dependency>
<groupId>org.apache.directory.server</groupId>
<artifactId>apacheds-core</artifactId>
<version>2.0.0-M15</version>
</dependency>
<dependency>
<groupId>org.apache.directory.server</groupId>
<artifactId>apacheds-all</artifactId>
<version>1.5.7</version>
</dependency>
<dependency>
<groupId>org.apache.directory.server</groupId>
<artifactId>apacheds-server-integ</artifactId>
<version>1.5.7</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>org.apache.directory.client.ldap</groupId>
<artifactId>ldap-client-api</artifactId>
<version>0.1</version>
</dependency>
Then I am running test from example i am receiving such error:
Exception in thread "main" java.lang.IncompatibleClassChangeError: Class org.apache.directory.server.core.DefaultDirectoryService does not implement the requested interface org.apache.directory.server.core.DirectoryService
at org.apache.directory.server.core.integ.FrameworkRunner.run(FrameworkRunner.java:170)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
Any ideas?
Test Class: http://pastebin.com/aPisMRSx
Change the version of apacheds-all to the same version as apacheds-core