Kafka Producer Class Not Found Exception - java

I tried to implement a simple producer consumer example with kafka and I achieved with the following properties:
Properties configProperties = new Properties();
configProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"localhost:" + portNumber);
configProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,"org.apache.kafka.common.serialization.ByteArraySerializer");
configProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,"org.apache.kafka.common.serialization.StringSerializer");
// Belirtilen property ayarlarına sahip kafka producer oluşturulur
org.apache.kafka.clients.producer.Producer producer = new KafkaProducer(configProperties);
However when I try the exact same configs, and everything else the same, in another project which is a plugin for a data visualization software, I got this error:
.... // Here there is some other stuff but I thing the important one is the below one
Caused by: java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/Producer
at App.MyControlPanel.<init>(MyControlPanel.java:130)
at App.CytoVisProject.<init>(CytoVisProject.java:29)
... 96 more
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.clients.producer.Producer
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 98 more
In the first one that I said it worked, I was using "mvn clean compile assembly:single", but in the second one I created a jar file for the whole project. Because the visualization software wants a jar file to install the plugin. Since every thing is same (At least I could not find any difference, I used same code) I guess the problem is about the way build the project. What happened here? What is the difference between "mvn clean compile assembly:single" and building a jar file in IntelliJ? Why I got this error and how to fix this? Thanks a lot for help!
As I said in the last comment of the first answer, I have a plugin which has manifest and transform as goal. Here:
<plugin>
<groupId>com.springsource.bundlor</groupId>
<artifactId>com.springsource.bundlor.maven</artifactId>
<version>1.0.0.M2</version>
<configuration>
<outputManifest>C:\Users\USER\AppData\Local\Temp\archetype2tmp/META-INF/MANIFEST.MF</outputManifest>
<failOnWarnings>false</failOnWarnings>
<removeNullHeaders>true</removeNullHeaders>
<manifestHeaders><![CDATA[Bundle-ManifestVersion: 2
Bundle-Name: CytoVisProject
Bundle-SymbolicName: CytoVisProject
Spring-DM-Version: ${pom.version}
]]></manifestHeaders>
</configuration>
<!-- generate the manifest automatically during packaging -->
<executions>
<execution>
<id>bundle-manifest</id>
<phase>package</phase>
<goals>
<goal>manifest</goal>
<goal>transform</goal>
</goals>
</execution>
</executions>
</plugin>
If I use a shade plugin like below:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.0</version>
<configuration>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
It does not works, because I need to use manifest and transform as goals in my plugin. How can I add kafka classes to the jar file that IntelliJ creates to solve this problem (I am not sure if this can solve or not)?

I've found an easier solution. I have changed
kafkaProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,"org.apache.kafka.common.serialization.StringSerializer");
to this
kafkaProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
And afterwards my code run, also as part of a pre-compiled plug-in.

What the problem is
At compile time kafka-clients-0.10.0.0.jar is available, so the code compiles successfully, but at runtime it is missing, hence the error you get.
How to fix it
You have 2 options:
Include the kafka JAR inside your JAR
Add the kafka JAR to your plugin's classpath

Also, make sure you don't have a dependency for kafka-clients marked with provided scope.

Related

ClassNotFoundEcception when running jar through Terminal (or command prompt)

So I know there are tons of questions like this one, and I have been through all of them, but I can't seem to find one like mine.
Basically, I have a java project with a lot of Maven Dependencies. The project compiles and runs just fine when I run it with IntelliJ, but now I am trying to run it through the terminal (or command prompt). In order to do that, I ran mvn package so I can get a jar file and when I run java -jar server.jar, I get the classic ClassNotFoundEcception exception. In my case, it says that it is:
Caused by: java.lang.ClassNotFoundException: org.apache.commons.fileupload.FileItemFactory
and when I suppress it by (temporarily) commenting out the part of the code that uses this class, I end up with the same error for another class. At this point, I know that I need to have some sort of folder (correct me if I'm wrong, but I believe it is called CLASSPATH) which contains the .jar of each dependency. Can anyone explain to me the situation in a clear way and how am I supposed to organize the .jar file of each dependency (if that is even what I have to do to fix my error).
The org.apache.commons.fileupload.FileItemFactory and most likely other dependencies are not in the classpath. Intellij includes dependencies in the classpath automatically, and this is the reason it works.
To run the program from CMD with all your depedecies included define the following plugin in pom.xml:
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.1.1</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
and run:
mvn clean package
java -jar server-jar-with-dependencies.jar

Generated Sources Root gone after every git pull in IntelliJ

One of the many modules I'm working on uses "generated-sources" for a couple auto-generated Java classes that the rest of the code depends on. Unfortunately, every single time I do a git pull it gets reset and I have to do mark the folder as "Generated Sources Root" again.
It's not a deal breaker, but it's really annoying. Isn't there a way to automate this? I don't know, some setting in IntelliJ or perhaps even directly in the pom.xml?
You can give a try to build-helper-maven-plugin and add more source directories to your project explicitly:
<project>
...
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>${project.build.directory}/generated-sources/...</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Also there is a little trick - actual source generation must happens before this goal execution.
In our project IDEA works pretty well with this approach.
Try adding generated-sources folder to git ignore file. Git ignore file if not already exists can be added in intelli idea like:
Then add generated-sources folder relative path like generated-sources/in git ignore file & commit .gitignore file. From next time any changes in that folder will not be overwritten or will be committed to server.

WAR bundles SNAPSHOT JARs with timestamps

I have this multi module project setup which uses Webstart and I need to bundle the WAR with SNAPSHOT JARs. When the JARs are bundled into the WAR, they are appended with a timestamp instead of the actual name. This is causing issues during their download.
Expected - ABC-1.0-SNAPSHOT.jar
Actual - ABC-1.0-20141002.211448-2.jar
Env:
OS: Unix
Maven: 3.2.1
JDK: 1.7
I have already tried useUniqueVersions=false by defining a maven-war-plugin and setting this in the manifest configuration.
My webstart config:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>webstart-maven-plugin</artifactId>
<version>1.0-beta-6</version>
<executions>
<execution>
<phase>process-resources</phase>
<goals>
<goal>jnlp-download-servlet</goal>
</goals>
</execution>
</executions>
<configuration>
<outputDirectoryName>.</outputDirectoryName>
<excludeTransitive>true</excludeTransitive>
<commonJarResources>
<jarResource>
...
</jarResource>
</commonJarResources>
<jnlpFiles>
<jnlpFile>
<templateFilename>JNLP-INF/APPLICATION_TEMPLATE.JNLP</templateFilename>
<outputFilename>client.jnlp</outputFilename>
<jarResources>
<jarResource>
...
</jarResource>
</jarResources>
</jnlpFile>
</jnlpFiles>
<sign>
...
</sign>
<outputJarVersions>false</outputJarVersions>
</configuration>
Appreciate any inputs.
UPDATE
Adding details about the WAR plugin I added
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.4</version>
</plugin>
The behavior continues. I read that maven 3 uses a unique snapshot system. But I am trying to work my way around it.
Also tried the following without any luck
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.4</version>
<configuration>
<outputFileNameMapping>${artifact.artifactId}-${artifact.baseVersion}.${artifact.extension}</outputFileNameMapping>
</configuration>
</plugin>
The creators of the maven-webstart-plugin have accidentally inverted the meaning of the flag useUniqueVersions. When set to true it produces jars of the form -SNAPSHOT.jar and when set to false (the default) it produces jars of the form --.jar
I found that setting the useUniqueVersions flag to true in the maven pom configuration produced my desired results.
I needed it like this because when deployed with the datestamp version of the snapshots when trying to launch it I was getting a "Resource not found" error when it tried to get the snapshot jar it was trying to launch. Presumably this means that the servlet code itself is not set up to handle the different naming on disk, despite the fact that the specific datestamp version number does end up in the jnlp itself.

Managing JAXB-generated classes in a Maven project

I have a Maven-based project, in which I trying to add some JAXB classes automatically generated by the "jaxb2-maven-plugin" Maven plugin. However, my first cut has me in a circular dependency loop:
Because these JAXB classes aren't generated yet, my other sources which reference them have compilation errors.
Because those other sources have compilation errors, these JAXB classes don't get generated.
It seems like there are two obvious possibilities for solving this:
Comment-out the broken references, so that the project builds and the JAXB classes are automatically generated. Then copy those generated sources from /target into /src/main/java, so that references to them won't cause compilation errors.
Create an entirely separate project, consisting of nothing but the JAXB stuff. Include it as a dependency in my main project.
Am I missing something here? Option #1 seems flat-out ridiculous... that just can't be the manner in which people use JAXB. Option #2 seems more rational, but still rather inefficient and cumbersome. I really have to take on the overhead of an entirely separate project just to use JAXB?
Are there any more elegant approaches that developers use to reference JAXB-generated classes in the same project where the Maven plugin generates them?
UPDATE: By request, here is the relevant portion of my POM:
<build>
<plugins>
<plugin>
<!-- configure the compiler to compile to Java 1.6 -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>jaxb2-maven-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>xjc</goal>
</goals>
</execution>
</executions>
<configuration>
<!-- The name of your generated source package -->
<packageName>com.mypackage</packageName>
</configuration>
</plugin>
</plugins>
</build>
When I run mvn clean package, I DO see my JAXB sources being generated beneath the /target subdirectory. However, those generated sources are not being automatically added to the classpath for the compile phase.
POST-RESOLUTION UPDATE: It turns out that my compilation issues had more to do with the fact that I was running in Eclipse, and its Maven integration has some issues with "jaxb2-maven-plugin". See this StackOverflow question for more detail on that issue and its resolution.
How did you configure your jaxb maven plugin? Normally it runs in the generate-sources lifecycle, which comes before the compile lifecycle. So your JAXB generated classes should already be there when your own code gets compiled, Maven puts them in target/generated-source and puts that folder on the classpath.
Edit:
This is my code we use at work (and which works as expected):
<plugin>
<groupId>com.sun.tools.xjc.maven2</groupId>
<artifactId>maven-jaxb-plugin</artifactId>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>generate</goal>
</goals>
</execution>
</executions>
<configuration>
<schemaDirectory>src/main/resources/<companyname>/xsd</schemaDirectory>
<includeSchemas>
<includeSchema>retrieval.xsd</includeSchema>
<includeSchema>storage.xsd</includeSchema>
</includeSchemas>
</configuration>
</plugin>
Apparently we use yet another jaxb plugin... (see also this thread: Difference of Maven JAXB plugins).
i would suggest you to split jaxb-generated classes (api) and your BL classes (implementation) to 2 maven projects with separate pom.xml for each, and the main root pom.xml with the compilation order. that way, you will be able to build api.jar, then maven will install it inside the local repo, and after that you can use it as the dependency of your implementation. so it will looks like:
-API\
--pom.xml - for api, jaxb generation
-IMPL\
--pom.xml - for impl, api dependency is here
pom.xml - main pom.xml with references to the projects above
Maybe try using the maven-jaxb2-plugin instead:
<plugin>
<groupId>org.jvnet.jaxb2.maven2</groupId>
<artifactId>maven-jaxb2-plugin</artifactId>
<version>0.8.2</version>
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
</execution>
</executions>
</plugin>
The answer from dfuse is correct, though. Either plugin should generate sources before compiling, and the result of the source generation will be on the classpath. I tested this with both plugins. Is it possible for you to post your schema, or at least the schema for the type that your code is failing to pick up on the classpath?

Eclipse Maven Plugin fails to create groovy-maven-archetype project

I have installed the Maven for Eclipse plugin from Sonatype.
(update site: http://m2eclipse.sonatype.org/update/)
I am creating a Maven project, and choosing to use the groovy-maven-archetype as my starting point.
However, halfway through, I am seeing:
04/03/09 12:52:28 GMT: [FATAL ERROR]
org.codehaus.mojo.groovy.stubgen.GenerateStubsMojo#execute()
caused a linkage error (java.lang.NoSuchMethodError). Check the realms:
... snip ...
Realm ID: plexus.core
org.codehaus.plexus.PlexusContainer.createChildContainer
(Ljava/lang/String;Ljava/util/List;Ljava/util/Map;)
Lorg/codehaus/plexus/PlexusContainer;
How can I fix this?
At a command prompt, enter this: mvn archetype:generate
Then, choose 40 (gmaven-archetype-basic)
Then, follow the prompts.
Once you have a maven project, you can enable Eclipse support by saying: mvn eclipse:eclipse
You can read Building Groovy Projects for more information.
Seems like a versioning problem to me. Are you sure you used all the right versions of the jars?
Getting Groovy-Eclipse, gmaven, and Eclipse all working together seems to be pretty tricky at the present. Once you've got a project created with mvn archetype:generate, as AWhitford mentions, this site will show you a few of the tweaks you'll need to make it work.
GMaven's stub creation for Java files interferes with Groovy-Eclipse, hence the section on that page about commenting out stub creation. However, I went with the method mentioned in the comments for the relevant bug (GMAVEN-61) and created multiple executions for the gmaven plugin, like so:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.groovy.maven</groupId>
<artifactId>gmaven-plugin</artifactId>
<version>1.0-rc-3</version>
<!-- http://jira.codehaus.org/browse/GMAVEN-61 -->
<executions>
<execution>
<id>default-cli</id>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
<execution>
<id>stubsonly</id>
<goals>
<goal>generateStubs</goal>
<goal>generateTestStubs</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
I'm still not certain myself that this is clean for both pure Maven usage as well as within Eclipse, but it at least got me to the point where I stopped spending hours trying to get anything to work and got me coding on my actual project.
The Groovy-Eclipse and GMaven documentation are good reading for background info.

Categories