We Keep Coding

java.lang.NoClassDefFoundError when trying to load class from JAR

I am working on a project that is supposed to parse texts from PDF files.
Having multiple dependencies I have decided to build a combined JAR with all the dependencies and the classes.
However, when I build JAR including dependencies via Intellij IDEA even though the JAR file is added properly and I can import the class the program throws NoClassDefFoundError (Please refer to the screenshot).
Firstly, I thought the jar wasn't in the classpath. However, even if I add -cp TessaractPDF.jar through VM Options the class still get undetected.
I think it is worth to mention that, everything works smoothly if I build JAR without dependencies and add the dependencies manually.
What should I do?
Exception in thread "main" java.lang.NoClassDefFoundError: me/afifaniks/parsers/TessPDFParser
at Test.main(Test.java:20)
Caused by: java.lang.ClassNotFoundException: me.afifaniks.parsers.TessPDFParser
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 1 more
Code Snippet:
import me.afifaniks.parsers.TessPDFParser;
import java.io.IOException;
import java.util.HashMap;
public class Test {
public static void main(String[] args) throws IOException {
System.out.println(System.getProperty("java.classpath"));
HashMap<String, Object> arguments = new HashMap<>();
arguments.put("imageMode", "binary");
arguments.put("toFile", false);
arguments.put("tessDataPath", "/home/afif/Desktop/PDFParser/tessdata");
TessPDFParser pdfParser = new TessPDFParser("hiers15.pdf", arguments);
String text = (String) pdfParser.convert();
System.out.println(text);
}
}

Getting an UnsatisfiedLinkError (undefined symbol) in Java while loading JNI dependencies even after successfully loading the required object file

I'm using Google OR-tools library (v6.4) for a project (though my question is not specific to this library). This consists of one jar, which has a few native dependencies (a bunch of ".so"/".dylib" object files, depending on the OS). This build for my project is being made on Ubuntu 14.04
The problem I'm facing: On trying to load a specific object file at runtime (using System.load()), I'm getting an UnsatisfiedLinkError with the message as "undefined symbol" (I've added the stacktrace below). However, I am loading the object file defining this symbol just before this, so I'm not sure why this error is being thrown.
I'm loading the dependencies in the following way: The object files are being packed into the jar created by Maven during build, and are being extracted and loaded (using System.load()) at runtime. The method for that is as follows:
public class EnvironmentUtils {
public static void loadResourceFromJar(String prefix, String suffix) {
String tempFilesDirectory = System.getProperty("java.io.tmpdir");
File tempFile = null;
try {
tempFile = new File(tempFilesDirectory + "/" + prefix + suffix);
tempFile.deleteOnExit();
try (final InputStream inputStream = EnvironmentUtils.class.getClassLoader().
getResourceAsStream(prefix+suffix)) {
if (inputStream == null) {
throw new RuntimeException(prefix + suffix + " was not found inside JAR.");
} else {
Files.copy(inputStream, tempFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
}
}
System.load(tempFile.getAbsolutePath());
} catch (Exception e) {
//Log top 10 lines of stack trace
}
}
}
This method is being called inside a static block for all dependencies:
public class DummyClass {
static {
String sharedLibraryExtension = EnvironmentUtils.getSharedLibraryExtension(); //.so for linux, .dylib for Mac
String jniLibraryExtension = EnvironmentUtils.getJniLibraryExtension(); //.so for linux, .jnilib for Mac
EnvironmentUtils.loadResourceFromJar("libfap", sharedLibraryExtension);
EnvironmentUtils.loadResourceFromJar("libcvrptw_lib", sharedLibraryExtension);
EnvironmentUtils.loadResourceFromJar("libortools", sharedLibraryExtension);
EnvironmentUtils.loadResourceFromJar("libdimacs", sharedLibraryExtension);
EnvironmentUtils.loadResourceFromJar("libjniortools", jniLibraryExtension);
}
}
On running System.load() for libdimacs.so, an UnsatisfiedLinkError is thrown. Stacktrace:
java.lang.UnsatisfiedLinkError: /tmp/libdimacs.so: /tmp/libdimacs.so: undefined symbol: _ZN6google14FlagRegistererC1IbEEPKcS3_S3_PT_S5_
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at com.(PROJECT_NAME).utils.EnvironmentUtils.loadResourceFromJar(EnvironmentUtils.java:78)
at com.(PROJECT_NAME).DummyClass.<clinit>(DummyClass.java:28)
However, this symbol "_ZN6google14FlagRegistererC1IbEEPKcS3_S3_PT_S5_" is present in libortools.so, which is being loaded before libdimacs. I verified this by running the following command:
objdump -t (LIBRARY_PATH)/libortools.so | grep _ZN6google14FlagRegistererC1IbEEPKcS3_S3_PT_S5_
This gave me the following output:
0000000000ce12cc gw F .text 00000091 _ZN6google14FlagRegistererC1IbEEPKcS3_S3_PT_S5_
So it would seem that the symbol should have been defined at the time of the System.load() call, unless there was some issue in loading the containing object file. To check if the object file had been loaded correctly, I used the approach detailed in this solution. Apart from the class detailed in that answer, I added the following lines after System.load() call in EnvironmentUtils.loadResourceFromJar() to print the most recently loaded library name:
public class EnvironmentUtils {
public static void loadResourceFromJar(String prefix, String suffix) {
...
System.load(tempFile.getAbsolutePath());
final String[] libraries = ClassScope.getLoadedLibraries(ClassLoader.getSystemClassLoader());
System.out.println(libraries[libraries.length - 1]);
}
}
The output (till just before the UnsatisfiedLinkError) is as follows:
/tmp/libfap.so
/tmp/libcvrptw_lib.so
/tmp/libortools.so
So libortools.so seems to be loading correctly, which means the symbol should be loaded in memory. The exact same code is working perfectly with the corresponding Mac (".dylib") dependencies (Built on MacOS Sierra 10.12.5). Would appreciate any advice on resolving this. Thank you.

I'm apologize that the java artifact may be broken currently...
you can use c++filt to demangle the symbol ;)
c++filt _ZN6google14FlagRegistererC1IbEEPKcS3_S3_PT_S5_
google::FlagRegisterer::FlagRegisterer<bool>(char const*, char const*, char const*, bool*, bool*)
In fact gflag has recently change its namespace from google:: to gflags:: and glog or protobobuf? try to find the correct one and I guess it failed...
note: Still not completely sure whose is the bad guy who use the google:: namespace since libortools merge all its static dependencies but I guess now you understand the bug...
note2: I have a patch in mizux/shared branch https://github.com/google/or-tools/commit/805bc0600f4b5645114da704a0eb04a0b1058e28#diff-e8590fe6fb5044985c8bf8c9e73c0d88R114
warning: this branch is currently broken and not ready yet. I'm trying ,for unix, to move from static to dynamic dependencies, so I need to fix all rpath, transitives deps etc... and in the process I also had to fix this issue (that I didn't reproduced while using static dependencies)
If too long to finish (we should create a release 6.7.2 or 6.8 (i.e. new artifact) by the end of May 2018) which maybe only contains this fix and not my branch...

cmu sphinx4 java - Runtime exception caused by FileNotFoundException

I have recently made a Java project with Sphinx4. I found this code online, and I slimmed it down to this to test if Sphinx4 was working:
public class App
{
private static final String ACOUSTIC_MODEL =
"resource:/edu/cmu/sphinx/models/en-us/en-us";
private static final String DICTIONARY_PATH =
"resource:/edu/cmu/sphinx/models/en-us/cmudict-en-us.dict";
public static void main(String[] args) throws Exception {
Configuration configuration = new Configuration();
configuration.setAcousticModelPath(ACOUSTIC_MODEL);
configuration.setDictionaryPath(DICTIONARY_PATH);
configuration.setGrammarName("dialog");
LiveSpeechRecognizer jsgfRecognizer =
new LiveSpeechRecognizer(configuration);
jsgfRecognizer.startRecognition(true);
while (true) {
String utterance = jsgfRecognizer.getResult().getHypothesis();
if (utterance.startsWith("hello")) {
System.out.println("Hello back!");
}
else if (utterance.startsWith("exit")) {
break;
}
}
jsgfRecognizer.stopRecognition();
}
}
However, it gave me this error:
Exception in thread "main" java.lang.RuntimeException: Allocation of search manager resources failed
at edu.cmu.sphinx.decoder.search.WordPruningBreadthFirstSearchManager.allocate(WordPruningBreadthFirstSearchManager.java:247)
at edu.cmu.sphinx.decoder.AbstractDecoder.allocate(AbstractDecoder.java:103)
at edu.cmu.sphinx.recognizer.Recognizer.allocate(Recognizer.java:164)
at edu.cmu.sphinx.api.LiveSpeechRecognizer.startRecognition(LiveSpeechRecognizer.java:47)
at com.weebly.controllingyourcomputer.bartimaeus.App.main(App.java:27)
Caused by: java.io.FileNotFoundException:
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
at java.net.URL.openStream(URL.java:1038)
at edu.cmu.sphinx.linguist.language.ngram.SimpleNGramModel.open(SimpleNGramModel.java:403)
at edu.cmu.sphinx.linguist.language.ngram.SimpleNGramModel.load(SimpleNGramModel.java:277)
at edu.cmu.sphinx.linguist.language.ngram.SimpleNGramModel.allocate(SimpleNGramModel.java:114)
at edu.cmu.sphinx.linguist.lextree.LexTreeLinguist.allocate(LexTreeLinguist.java:334)
at edu.cmu.sphinx.decoder.search.WordPruningBreadthFirstSearchManager.allocate(WordPruningBreadthFirstSearchManager.java:243)
... 4 more
I thought it might be something about it not being able to find the paths for ACOUSTIC_MODEL or DICTIONARY_PATH, so I changed the resource: strings to things like %HOME%\\Downloads\\sphinx4-5prealpha-src\\sphinx4-5prealpha-src\\sphinx4-data\\src\\main\\resources\\edu\\cmu\\sphinx\\models\\en-us or paths with forward slashes or with C:\Users\Username\... but none of the paths worked. I know the paths exist because I copy and pasted them from the properties window of the actual resources.
So my question is: is it some of the code that I deleted from the original source code that is causing this error, is it something wrong with the paths, or is it entirely different?
EDIT
By the way, I am using Maven to build my project. I added the dependencies specified on the Sphinx4 website to my pom.xml, but it didn't work (it didn't recognize imports such as edu.com.sphinx.xxx) so I downloaded the JARs from the website they said to download them from and added them to my projects "Libraries" in my Java Build Path in Eclipse.

is it some of the code that I deleted from the original source code that
is causing this error
Yes, you deleted too much.
To recognize with grammar you need to make three calls:
configuration.setGrammarPath(GRAMMAR_PATH);
configuration.setGrammarName(GRAMMAR_NAME);
configuration.setUseGrammar(true);

Executing Sample Flink Program in Local

I am trying to execute a sample program in Apache Flink in local mode.
import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.util.Collector;
public class WordCountExample {
public static void main(String[] args) throws Exception {
final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
DataSet<String> text = env.fromElements(
"Who's there?",
"I think I hear them. Stand, ho! Who's there?");
//DataSet<String> text1 = env.readTextFile(args[0]);
DataSet<Tuple2<String, Integer>> wordCounts = text
.flatMap(new LineSplitter())
.groupBy(0)
.sum(1);
wordCounts.print();
env.execute();
env.execute("Word Count Example");
}
public static class LineSplitter implements FlatMapFunction<String, Tuple2<String, Integer>> {
#Override
public void flatMap(String line, Collector<Tuple2<String, Integer>> out) {
for (String word : line.split(" ")) {
out.collect(new Tuple2<String, Integer>(word, 1));
}
}
}
}
It is giving me exception :
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/InputFormat
at WordCountExample.main(WordCountExample.java:10)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.InputFormat
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 1 more
What am I doing wrong?
I have used the correct jars also.
flink-java-0.9.0-milestone-1.jar
flink-clients-0.9.0-milestone-1.jar
flink-core-0.9.0-milestone-1.jar

Adding the three Flink Jar files as dependencies in your project is not enough because they have other transitive dependencies, for example on Hadoop.
The easiest way to get a working setup to develop (and locally execute) Flink programs is to follow the quickstart guide which uses a Maven archetype to configure a Maven project. This Maven project can be imported into your IDE.

NoClassDefFoundError extends LinkageError
Thrown if the Java Virtual Machine or a ClassLoader instance tries to
load in the definition of a class (as part of a normal method call or
as part of creating a new instance using the new expression) and no
definition of the class could be found. The searched-for class
definition existed when the currently executing class was compiled,
but the definition can no longer be found.
Your code/jar dependent to hadoop. Found it here download jar file and add it in your classpath org.apache.hadoop.mapreduce.InputFormat

Firstly, the flink jar files which you have included in your project are not enough, include all the jar files which are present in the lib folder present under the flink's source folder.
Secondly, " env.execute();
env.execute("Word Count Example");" These lines of code are not required since you are just printing your dataset onto the console; you're not writing the output into a file(.txt, .csv etc.). So, better to remove these lines (Sometimes throws errors if included in code if not required (observed a lot of times))
Thirdly, while exporting the jar files for your Java Project from your IDE, don't forget to select your 'Main' class.
Hopefully, after making the above changes, your code works.

Error using sphinx4 jars without Maven

I have a problem with the API Sphinx4 and I can't figure out why it doesn't work.
I try to write a little class for capture the voice of an user and write his speaking on a file.
1) I have create a new java project on Eclispe.
2) I have create the class TranscriberDemo.
3) I have create a folder "file".
4) I have copy the folder "en-us" and the files "cmudict-en-us.dict", "en-us.lm.dmp", "10001-90210-01803.wav" on the folder "file".
5) I don't use maven, so I have just include the jar files "sphinx4-core-1.0-SNAPSHOT.jar" and "sphinx4-data-1.0-SNAPSHOT.jar".
you can download them here:
core: https://1fichier.com/?f3y6vqupdr
data: https://1fichier.com/?lpzz8jyerv
I know that the source code is available
here: https://github.com/erka/sphinx-java-api
or here: http://sourceforge.net/projects/cmusphinx/files/sphinx4
But I don't use maven so I can't compile them.
My class:
import java.io.InputStream;
import edu.cmu.sphinx.api.Configuration;
import edu.cmu.sphinx.api.SpeechResult;
import edu.cmu.sphinx.api.StreamSpeechRecognizer;
import edu.cmu.sphinx.result.WordResult;
public class TranscriberDemo
{
public static void main(String[] args) throws Exception
{
System.out.println("Loading models...");
Configuration configuration = new Configuration();
// Load model from the jar
configuration.setAcousticModelPath("file:en-us");
configuration.setDictionaryPath("file:cmudict-en-us.dict");
configuration.setLanguageModelPath("file:en-us.lm.dmp");
StreamSpeechRecognizer recognizer = new StreamSpeechRecognizer(configuration);
InputStream stream = TranscriberDemo.class.getResourceAsStream("file:10001-90210-01803.wav");
stream.skip(44);
// Simple recognition with generic model
recognizer.startRecognition(stream);
SpeechResult result;
while ((result = recognizer.getResult()) != null)
{
System.out.format("Hypothesis: %s\n", result.getHypothesis());
System.out.println("List of recognized words and their times:");
for (WordResult r : result.getWords())
{
System.out.println(r);
}
System.out.println("Best 3 hypothesis:");
for (String s : result.getNbest(3))
System.out.println(s);
}
recognizer.stopRecognition();
}
}
My log:
Loading models...
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/base/Function
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:191)
at edu.cmu.sphinx.util.props.ConfigurationManager.getPropertySheet(ConfigurationManager.java:91)
at edu.cmu.sphinx.util.props.ConfigurationManagerUtils.listAllsPropNames(ConfigurationManagerUtils.java:556)
at edu.cmu.sphinx.util.props.ConfigurationManagerUtils.setProperty(ConfigurationManagerUtils.java:609)
at edu.cmu.sphinx.api.Context.setLocalProperty(Context.java:198)
at edu.cmu.sphinx.api.Context.setAcousticModel(Context.java:88)
at edu.cmu.sphinx.api.Context.<init>(Context.java:61)
at edu.cmu.sphinx.api.Context.<init>(Context.java:44)
at edu.cmu.sphinx.api.AbstractSpeechRecognizer.<init>(AbstractSpeechRecognizer.java:37)
at edu.cmu.sphinx.api.StreamSpeechRecognizer.<init>(StreamSpeechRecognizer.java:35)
at TranscriberDemo.main(TranscriberDemo.java:27)
Caused by: java.lang.ClassNotFoundException: com.google.common.base.Function
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 12 more
Thanks for your help =)

There are multiple issues with your code and your actions:
3) I have create a folder "file".
Not needed
4) I have copy the folder "en-us" and the files "cmudict-en-us.dict", "en-us.lm.dmp", "10001-90210-01803.wav" on the folder "file".
Not needed, you already have models as part of sphinx4-data package.
5) I don't use maven, so I have just include the jar files "sphinx4-core-1.0-SNAPSHOT.jar" and "sphinx4-data-1.0-SNAPSHOT.jar".
This is very wrong because you took outdated jars from unauthorized location. The right place to download jars is listed in tutorial http://oss.sonatype.org
https://oss.sonatype.org/service/local/repositories/snapshots/content/edu/cmu/sphinx/sphinx4-core/1.0-SNAPSHOT/sphinx4-core-1.0-20150223.210646-7.jar
https://oss.sonatype.org/service/local/repositories/snapshots/content/edu/cmu/sphinx/sphinx4-data/1.0-SNAPSHOT/sphinx4-data-1.0-20150223.210601-7.jar
You took malicious jars from some random website which might have a virus or rootkit in them.
here: https://github.com/erka/sphinx-java-api
This is a wrong link too. The correct link is http://github.com/cmusphinx/sphinx4
InputStream stream = TranscriberDemo.class.getResourceAsStream("file:10001-90210-01803.wav");
Here you use file: URL scheme which points to files in inappropriate context. If you want to create InputStream from file do like this:
InputStream stream = new FileInputStream(new File("10001-90210-01803.wav"));
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/base/Function
This error is caused by the fact you took a jar from other place and it said you need additional dependencies. When you see ClassDefFoundError it means you need to add additional jar into your classpath. With official sphinx4 you should not see this error.

Solved.
In fact it was a silly mistake...
Thank you #Nikolay for your answer. I already accept your answer but I resume the process here:
1) Download the sphinx4-core and sphinx4-data jars from https://oss.sonatype.org/#nexus-search;quick~sphinx4.
2) Include them in your project.
3) Test your code.
import edu.cmu.sphinx.api.Configuration;
import edu.cmu.sphinx.api.LiveSpeechRecognizer;
import edu.cmu.sphinx.api.SpeechResult;
public class SpeechToText
{
public static void main(String[] args) throws Exception
{
Configuration configuration = new Configuration();
configuration.setAcousticModelPath("resource:/edu/cmu/sphinx/models/en-us/en-us");
configuration.setDictionaryPath("resource:/edu/cmu/sphinx/models/en-us/cmudict-en-us.dict");
configuration.setLanguageModelPath("resource:/edu/cmu/sphinx/models/en-us/en-us.lm.dmp");
LiveSpeechRecognizer recognizer = new LiveSpeechRecognizer(configuration);
recognizer.startRecognition(true);
SpeechResult result;
while ((result = recognizer.getResult()) != null)
{
System.out.println(result.getHypothesis());
}
recognizer.stopRecognition();
}
}
And that is all!
If you need the source code of Sphinx4: https://github.com/cmusphinx/sphinx4