I have exported two jars of two apps I have in the same Kafka/Spark Streaming project.
The jar with the Kafka Producer works fine. The jar with the Spark Consumer returns this error:
NoClassDefFoundError: kafka/serializer/StringDecoder
They share the same dependencies folder, which I have obviously exported in order to make the jars work outside Eclipse.
In Eclipse they both work fine.
How can I fix this?
You need Kafka itself in the class path, not just spark-streaming-kafka. In Eclipse you do, because your build system retrieves all dependencies, and Eclipse integrates with it.
I looked into the jar of spark-streaming-kafka and it contains the .class file that is named in the error: org.apache.spark.streaming.kafka.KafkaUtils$.createStream(KafkaUtils.scala:55). So I can't understand why he misses this class.
Presumably you mean that contains org.apache.spark.streaming.kafka.KafkaUtils$, but when JVM tries to load it, it discovers that it needs other classes, including kafka.serializer.StringDecoder, which is not there.
Related
I am trying to use a number of external libraries in a Java project. The project runs fine from Intelij but I want to package it in a jar, (or something else) so I can distribute it to others. When I package it in a jar it works if I just do hello world, but as soon as I start using my libraries I get the error bellow. I have also tried packaging it as an application but when I run the batch file it just opens and then immediatly closes a command window. I read all the other posts and nothing is fixing my problem. Please help
My error
My build.gradle
The problem is most likely that your classpath does not point to the correct relative location of your 3rd party libraries. You can check the manifest file to verify if the paths are correct.
However, if it is a runnable jar file with a main method (which it looks like it is), you should use the Application plugin and package it with the Distribution plugin. Right now you are using the Java Library Distribution plugin, which is for libraries. If you do this, you can remove most of the stuff under your jar task.
When testing it locally, run gradle run and when ready, use gradle distZip to create a zip of it all. It will create a script used to start the application with the correct classpath.
Alternatively, you could also package it in a fat jar using the Shadow plugin or similar.
I've created an application which has a few external libraries, i am trying to distribute to another machine. I want the application to work from just the .jar file.
I know its something to do with the 'Resolve Project Problems', is there a way to prevent this when distributing to another machine
I have already tried adding a library to the distribution folder, it carries over but doesn't do anything with it
Run the application on a new machine without having to 'Resolve project problems' through net beans
You will need to Create a FAT jar
Two Useful links which helped me where
How to include jars in lib into project jar file in Netbeans?
https://dzone.com/articles/java-chronicle-bytes-kicking-the-tires
This didn't entirely fix the issue, i was using a dynamic file string with in my program
I copied the data folder from my application and pasted it in the store folder and rebuilt and also repackaged the fat jar. This made it work
I have a Maven-Springboot project setup. After doing a mvn install I can run the jar file in command prompt using java -jar <my-jar-file.jar>
There was a dependency to a jar say as-common-1.0.0.jar that is there in my-jar-file.jar. Now I want to override the version of this dependent jar at run time by giving an external jar. something like:
java -jar my-jar-file.jar using as-common-1.0.1.jar
I went through many SO posts like thisinclude external jar when running java -jar but they didn't help.
is it achievable?
EDIT: The issue we have is many of our applications depend upon one of our internal framework jar which gets updated(version) often.
So every time changing the pom file and re-deploying all the apps doesn't look feasible to us. We want somehow the dependency of this particular jar is given at run time. Any idea regarding best possible way to manage this scenario?
No, this is not sensible. At best, you would load classes twice and this may/may not work.
If you build a complete jar containing everything, you cannot swap content at runtime. You need to rebuild it.
I am trying to tackle a persistent ClassNotFoundException. We have an application specific jar that calls a shared commons jar, and uses reflection create an object that we specify in an XML file. This is where it fails.
The application specific jar resides in my NSF. The commons jar resides on the server. I wish to keep the jars in the NSF, because I will be replacing them frequently and don't have access to the server directories, even in Unit.
We put the application jar in the same package as the commons and this didn't help. My suspicion is that the server jar cannot find the application jar. The application jar successfully calls the commons jar, so it is working in that direction.
I tried putting a copy of the commons jar in the NSF, and this didn't help. I think it is still running the server jar first. Jars have same name.
My question is: How do I force the Domino Server runtime to use the application's 'commons' jar instead of the server's 'commons' jar? The thinking being that the local commons jar will find the class of the application jar in the same package, and I can get past this exception.
UPDATE: This process runs when run from a jUnit test, and when a main method is created.
The key is going to be identifying if it's failing because of a security exception or just not finding the jar. If it hits a security exception loading the jar, it just gives that ClassNotFoundException. That can be because it's using the same ClassLoader as the XPpages runtime and was the problem I hit with apache.commons.lang.
If a jar is likely to change regularly and you want it from multiple NSFs, but don't have access to the server, the best option is to create an OSGi plugin (aka Extension Library). This will overcome most security exceptions as well as using a different classloader.
Put it in WebContent\lib, then right-click on it and choose Build Path > Add to Build Path. It will then use a different ClassLoader. See the comments on this blog post http://www.intec.co.uk/how-to-add-in-built-java-packages/. The lib folder is added by default in R9. You do need to add it to the build path, then it jumps into a Referenced Libraries folder.
I have a jar file that references 6 other jars through a manifest file.
I now, however, want to try and compile the jars into one jar file, reasons being: I want them to be cached so the applet does not have a long loading time as one of those jars are required, rather one big waiting time to download all required files and then store it in the cache.
I have tried including the referenced Jar files in the build, unfortunately, all this does is cause a java.lang.NoClassDefFoundError error, so I had to add a manifest file.
What is the best/easiest way to achieve this? I am using eclipse, and not building with ant.
Using eclipse while exporting you can create with the option of
Package required libraries into generate JAR
By doing this eclipse uses its own class loader and reference all the files which are packed in a single jar.