We Keep Coding

Executing Sample Flink Program in Local

I am trying to execute a sample program in Apache Flink in local mode.
import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.util.Collector;
public class WordCountExample {
public static void main(String[] args) throws Exception {
final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
DataSet<String> text = env.fromElements(
"Who's there?",
"I think I hear them. Stand, ho! Who's there?");
//DataSet<String> text1 = env.readTextFile(args[0]);
DataSet<Tuple2<String, Integer>> wordCounts = text
.flatMap(new LineSplitter())
.groupBy(0)
.sum(1);
wordCounts.print();
env.execute();
env.execute("Word Count Example");
}
public static class LineSplitter implements FlatMapFunction<String, Tuple2<String, Integer>> {
#Override
public void flatMap(String line, Collector<Tuple2<String, Integer>> out) {
for (String word : line.split(" ")) {
out.collect(new Tuple2<String, Integer>(word, 1));
}
}
}
}
It is giving me exception :
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/InputFormat
at WordCountExample.main(WordCountExample.java:10)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.InputFormat
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 1 more
What am I doing wrong?
I have used the correct jars also.
flink-java-0.9.0-milestone-1.jar
flink-clients-0.9.0-milestone-1.jar
flink-core-0.9.0-milestone-1.jar

Adding the three Flink Jar files as dependencies in your project is not enough because they have other transitive dependencies, for example on Hadoop.
The easiest way to get a working setup to develop (and locally execute) Flink programs is to follow the quickstart guide which uses a Maven archetype to configure a Maven project. This Maven project can be imported into your IDE.

NoClassDefFoundError extends LinkageError
Thrown if the Java Virtual Machine or a ClassLoader instance tries to
load in the definition of a class (as part of a normal method call or
as part of creating a new instance using the new expression) and no
definition of the class could be found. The searched-for class
definition existed when the currently executing class was compiled,
but the definition can no longer be found.
Your code/jar dependent to hadoop. Found it here download jar file and add it in your classpath org.apache.hadoop.mapreduce.InputFormat

Firstly, the flink jar files which you have included in your project are not enough, include all the jar files which are present in the lib folder present under the flink's source folder.
Secondly, " env.execute();
env.execute("Word Count Example");" These lines of code are not required since you are just printing your dataset onto the console; you're not writing the output into a file(.txt, .csv etc.). So, better to remove these lines (Sometimes throws errors if included in code if not required (observed a lot of times))
Thirdly, while exporting the jar files for your Java Project from your IDE, don't forget to select your 'Main' class.
Hopefully, after making the above changes, your code works.

java.lang.NoClassDefFoundError (Java, Eclipse, Fuse-JNA, Ubuntu)

via eclipse, I am trying to run builtin example of file system (HelloFS.java) of fuse-jna, but it gives me java.lang.NoClassDefFoundError .
My source project is in /home/syed/workspace/HelloFS
fuse-jna class files are in home/syed/Downloads/fuse-jna-master/build/classes/main/net/fusejna
In eclipse, I added class folder via buildpath and also jre path in envirnment file. I attached snapshot below.
Please help me run this example in eclipse.
error:
Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jna/Structure
at net.fusejna.FuseFilesystem.mount(FuseFilesystem.java:545)
at net.fusejna.FuseFilesystem.mount(FuseFilesystem.java:550)
at HelloFS.main(HelloFS.java:22)
Caused by: java.lang.ClassNotFoundException: com.sun.jna.Structure
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 3 more
here is code of builtin example file system (with not red underline, which i think means that eclipse build path is entered correctly, ):
import java.io.File;
import java.nio.ByteBuffer;
import net.fusejna.DirectoryFiller;
import net.fusejna.ErrorCodes;
import net.fusejna.FuseException;
import net.fusejna.StructFuseFileInfo.FileInfoWrapper;
import net.fusejna.StructStat.StatWrapper;
import net.fusejna.types.TypeMode.NodeType;
import net.fusejna.util.FuseFilesystemAdapterFull;
public class HelloFS extends FuseFilesystemAdapterFull
{
public static void main(String args[]) throws FuseException
{
/*if (args.length != 1) {
System.err.println("Usage: HelloFS <mountpoint>");
System.exit(1);
}*/
new HelloFS().log(true).mount("./testfs1");
}
private final String filename = "/hello.txt";
private final String contents = "Hello World!\n";
#Override
public int getattr(final String path, final StatWrapper stat)
{
if (path.equals(File.separator)) { // Root directory
stat.setMode(NodeType.DIRECTORY);
return 0;
}
if (path.equals(filename)) { // hello.txt
stat.setMode(NodeType.FILE).size(contents.length());
return 0;
}
return -ErrorCodes.ENOENT();
}
#Override
public int read(final String path, final ByteBuffer buffer, final long size, final long offset, final FileInfoWrapper info)
{
// Compute substring that we are being asked to read
final String s = contents.substring((int) offset,
(int) Math.max(offset, Math.min(contents.length() - offset, offset + size)));
buffer.put(s.getBytes());
return s.getBytes().length;
}
#Override
public int readdir(final String path, final DirectoryFiller filler)
{
filler.add(filename);
return 0;
}
}
This is envirnment file contents:
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
JAVA_HOME="/usr/lib/jvm/java-6-openjdk-i386"
This is fuse-jna classes path
I added /main folder
========================================================
#Viktor K. Thanks for the help,
the above mentioned error is fixed by downloading and adding com.sun.jna » jna to referece library
but now it shows me new error as
Dec 28, 2013 1:18:25 PM HelloFS getName
INFO: Method succeeded. Result: null
Dec 28, 2013 1:18:25 PM HelloFS getOptions
INFO: Method succeeded. Result: null
Exception in thread "main" java.lang.NoSuchMethodError: com.sun.jna.Platform.getOSType()I
at net.fusejna.Platform.init(Platform.java:39)
at net.fusejna.Platform.fuse(Platform.java:26)
at net.fusejna.FuseJna.init(FuseJna.java:113)
at net.fusejna.FuseJna.mount(FuseJna.java:172)
at net.fusejna.FuseFilesystem.mount(FuseFilesystem.java:545)
at net.fusejna.FuseFilesystem.mount(FuseFilesystem.java:550)
at HelloFS.main(HelloFS.java:22)
=======================================================
Hmmm
The one that I downloaded was not campatable I think,
in temp folder of fuse-jna
/home/syed/Downloads/fuse-jna-master/build/tmp/expandedArchives/jna-3.5.2.jar_r4n26u14up0smlb84ivcvfnke/
there was jna3.5.2 classes, I imported that to libraray, now its working fine.
My problem solved. Thanks a lot.

The problem is not in Fuse-JNA library. Fuse-JNA library is obviously dependent on jna library (can be found in public maven repository http://mvnrepository.com/artifact/com.sun.jna/jna). You need to add this library as dependency in your project. You can see that in your project's referenced libraries there is no com.sun.jna package available.
In general - if you want to use package A (in your case Fuse-JNA) and the package A depends on package B (in your case JNA) you have to add JNA package yourself as dependency to your project. In general it is very hard to find out what are all required dependencies of the packages that you want to use - therefore many projects are using maven (or any alternative like gradle). Check this if you want to learn more : Why maven? What are the benefits?. I strongly suggest to use a tool for dependency resolution (like maven) over manual dependency resolution.
Another approach is to download a fuse jar with all dependencies - if you believe that it is the only library you'll need. However, adding jar with dependencies can lead to a big disaster if you add later other dependencies. This could lead to dependencies conflict, which is hard to find problem.

why wrong name with NoClassDefFoundError

I created a List.java file in folder UtilityPack which contains this code
package Utilities;
public class List
{
private class node{}
public void insert(int data){}
public void print(){}
public static void main(String[] s){}
}
To compile i did
C:\UtilityPack>javac List.java
But when I try to run with
C:\UtilityPack>java -classpath . List
OR
C:\UtilityPack>java List
I get error
Exception in thread "main" java.lang.NoClassDefFoundError: List (wrong name: Uti
lities/List)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:14
2)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
I have been trying to execute this program from last 3 hours but nothing worked..please help

You need the fully qualified name e.g.
java -cp . Utilities.List
i.e. you're telling the JVM to look from the current direct (-cp .) for a class Utilities.List, which it will expect in the file Utilities\List.class.
To be more consistent you should put the .java file under a Utilities directory (yes - this is tautologous - the package specifies this, but it's consistent practise).
I would also avoid calling your class List. At some stage you're going to import a java.util.List and it'll all get very confusing!
Finally, as soon as you get more than a couple of classes, investigate ant or another build tool, and separate your source and target directories.

Use the complete name of the class to lauch your program :
java Utilities.List
But the folder name should also match the package name.

Your directory structure needs to follow your Java package pathing. IOW, if the class Listis in the package Utilities, you need to situate it in a directory called Utilities, which should be at the root level of your project, i.e. the path of the source file should be C:\UtilityPack\Utilities\List.java. When you are in C:\UtilityPack (project root), you compile and run List by referencing it as Utilities.List.
You might also consider using Eclipse, it will prevent this sort of things from happening, or any other Java IDE.

How do I resolve java.lang.ClassNotFoundException? [closed]

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
I am at a very introductory level of programming with java. I am writing a simple code that involves taking an investment and adding an intrest rate into it. The code is not finished yet but I hav run into the java.lang.ClassNotFoundException error. Since I am so new to java, I have not yet run into this problem before. Where my confusion comes in is the program will compile. I really don't know how to approach this problem.
As I said I dont have a clue about where to start on this, here is the error.
Exception in thread "main" java.lang.NoClassDefFoundError: CDCalc/java
Caused by: java.lang.ClassNotFoundException: CDCalc.java
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
The entire code
import java.util.Scanner;
public class CDCalc
{
public static void main(String args[])
{
int Count = 0;
int Investment = 0;
double Rate = 0;
Scanner userInput = new Scanner(System.in);
System.out.println("How much money do you want to invest?");
int Invest = userInput.nextInt();
System.out.println("How many years will your term be?");
double Term = userInput.nextInt();
System.out.println("Investing: " + Investment);
System.out.println(" Term: " + Term);
if (Term <= 1)
{
Rate = .3;
}
else if (Term <= 2)
{
Rate = .45;
}
else if (Term <= 3)
{
Rate = .95;
}
else if (Term <= 4)
{
Rate = 1.5;
}
else if (Term <= 5)
{
Rate = 1.8;
}
System.out.println("Total is: " + (Rate * Invest));
}
}
I would greatly appreciate any help on this. Thanks
edit
I apologize, I should have included this. The code did compile just fine, the problem came in when I ran it. I did use javav CDCalc.java and java CDCalc and thats when the error came up. The even stranger thing is I didn't change a thing, closed out terminal and my text editor, deleted the saved files, reopened everything, saved it, compiled it, and it runs fine now. I apologize again for this post but it seems it fixed itself! –

Your code should compile fine. The way you compile Java files is different than how you execute them.
You compile with
javac CDCalc.java
...and run them with
java CDCalc

Looks like you tried to run it as java CDCalc.java, but it should be just java CDCalc.
The Java command takes a class name (not a file name), so there is no ".java" at the end, and no slashes or backslashes but dots for the package name (your class does not have a package).

This is an sample example:
public class HelloWorldDemo {
public static void main(String args[]) {
System.out.println("Hello world test message");
}
}
This is sample hello world program,When i compile this program using javac HelloWorldDemo.java command, this compiles fine and generates HelloWorldDemo.class in the current directory,
After running this programm using java HelloWorldDemo command, I am getting the below exceptions.
Exception in thread "main" java.lang.NoClassFoundError: HelloWorldDemo
thread main throws this error and exit the program abnormally.
This reason for this error is java virtual machine can not find class file at run time. java command looks for the classes that are there in the current directory, so if your class file is not in current directory, you have to set in classpath, so the solution is to place this .class file in the classpath
classpath is the enviornment variable in every system which points to class files in the directories. if you classfile is in jar file, jar should be in classpath. classpath can be absolute(complete path) or relative path( related to directory )
solve java.lang.NoClassDefFoundError :-
HelloWorldDemo.class is not avialble at runtime, so we have to set the class file to java command using -classpath option
java -classpath . HelloWorld
This is for fixing NoClassDefFoundError error by setting classpath inline for java command.
We are instructing the jvm to look for the HelloWorldDemo.class in the current directory by specifying .
if class file is in different directory, we need specify the complete directory absolute or relative path instead of . for java command
Fix for java.lang.NoClassDefFoundError in windows:-
To solve NoClassDefFoundError error in windows , we have to set CLASSPATH environment variable.
to set classpath in windows, we have to configure the below values
set CLASSPATH=%CLASSPATH%;.;
%CLASSPATH% means existing classpath to be added and . points to current directory
After setting classpath,
java HelloWorldDemo
command works fine and prints hello world message

Your program seems fine. Did you compile your java file using Javac?
You need to perform steps below:
Compiling the .java file (source code) using javac
javac CDCalc.java
This should create a class file named CDCalc.class
Executing the compiled class file using java
java CDCalc

How to Use External Class Files in an Eclipse Project

My lecturer didn't provide us with the .java files for a tutorial. My question is, how would i use his class files in my eclipse project, and defeat the following error?
Error:
Exception in thread "main" java.lang.NoClassDefFoundError: lec/utils/InputReader
at randomIt.main(randomIt.java:17)
Caused by: java.lang.ClassNotFoundException: lec.utils.InputReader
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 1 more
Here is my code:
import java.util.Random;
import lec/utils.InputReader;
public class randomIt {
public static void main(String[] args) {
Random generator = new Random();
InputReader myReader = new InputReader();
//Pick a number randomly between 1 and 10!
int number = generator.nextInt(10)+1;
//Ask user to guess...!
System.out.println("Take a guess (1 to 10)");
if (number == myReader.readInt()){
System.out.println("You win");
}
else {
System.out.println("It was " + number + ", tough Luck");
}
}
And here is my Folder Structure:
Random /
*/ bin
* / lec / utils /InputReader
* / src / randomIt.java
Note: his class file is "InputReader.class"

I've had a play with Eclipse to work this one out. Give the following a go:
Create the following directory structure (your desktop will do) classes/lec/utils
Place the InputReader class file in the utils directory.
Remove any references you have to InputReader you currently have in your build path.
Using (right click on project) Properties->Java Build Path->Libraries select the 'Add external class folder' and select the 'classes' folder you created on your desktop and click OK.
Now in the 'Referenced Libraries' in the project folder you should have one called 'classes' and a package path under that called 'lec.utils' which contains the InputReader class.
You can use that class using 'import lec.utils.InputReader' in you own class.
Hope that Helps.

In the project configuration menu, there is a "Build Path->Configure Build Path" menu item. Within that, there is an option to add an "external class folder". Put all the provided class files in a folder, and add that folder to the build path.

You should make the following changes
Modify your randomIt class to have following include line (no lecs/ )
import utils.InputReader
Modify the filename as rnadmIt.java (and not randomit.java) . The class name and fie name has to be exactly same. Also per Sun convention the class should start with a capital letter
$ cd Random
$ javac -classpath ./lec src/randomIt.java