We Keep Coding

In Java, should I be creating a new Package, Folder, or Source Folder?

There are a couple of questions on SO that sort of hit this, but I am totally new to Java development and I don't know the correct way to approach this.
I have a C# solution, containing two projects (my app, and a unit test project) and within the app, most things are put into folders eg. Interfaces, Exceptions etc.
I am trying to recreate this in Java / Eclipse, but I don't know how. I ended up with lots of packages, which sounds really bad. I also tried adding a source folder but that ended up being outside of the package.
Could anyone point me in the right direction?
Namely, which of those should I use to represent my unit test project/set of unit tests, and subfolders which exist just for organising stuff.
Edit: It also says use of the default package is not advised. What should I be doing?
Edit 2: Here is what it looks like. Does this look vaguely correct? My original C# solution is on the right.

In a typical java eclipse project, you will have one or more source folders (for example one for app code, one for your unit tests).
Each folder contains a package tree, typically starting with your base package, for example com.mycompany.myapp.
In order to avoid name collisions, packages names are usually start with the domain name of the entity who is the author of the code, starting with the top-level-domain and going backwards (more general to more specific). That way, each class fully qualified name is unique. For example if google creates a class named List, it will be known as com.google.List, and it will not enter in conflict with the existing java.util.List interface.
You can have a unlimited number of packages inside this base package, for example :
com.mycompany.myapp.persistence
com.mycompany.myapp.domain
com.mycompany.myapp.services
com.mycompany.myapp.web
It all depends on your project and the way you want to organize your code and your classes.
At the logical level, packages are named with dots as separator. They contain java classes.
At the physical on disk level, each package is a directory. The java classes are contained in .java files (most frequently one class per file).
In Eclipse a "source folder" is a folder inside your project that is known to Eclipse to contain java source files. It will be compiled included in the output (for example JAR file) when you build your project.
In Eclipse, you usually view them at the logical level, showing packages. When you tell Eclipse to "create a new package", it will create the directory for you. For example, if you tell it to create the com.mycompany.myproject package, it will automatically create a com folder containing a mycompany folder containing a myproject folder.

In java source tree structure must match package structure
so foo.bar package must be laid out in
src/foo/bar
Also default package may not be advised - but you can still use it - better to put things in a package though

In java different project development structure are flowed according to type of project.
So as you are new to java and Eclipse so it's better to install maven plugin and create maven project and choose a archetypes according to your project type like a standalone or web based.
The maven plugin will create the project structure including packages,test packages source folder etc. You can get more about project structure from this

Using the default package may create namespace collisions. Imagine you're creating a library which contains a MyClass class. Someone uses your library in his project and also has a MyClass class in his default package. What should the compiler do? Package in Java is actually a namespace which fully identifies your project. So it's important to not use the default package in the real world projects.

Java packages and compilation (why, not how)

I'm working on some Java code in eclipse. Code is contained in a single class called Adder, which in Eclipse is in the package org.processing. The first thing in the class file is the line
package org.processing
Q1) What, exactly is this line doing? Why is there, what's it's role.
The code runs fine in eclipse, however, when I move into the workspace if I go to the src/org/processing/ folder in src, compile with javac Adder.class when I try and run using java Adder I get the following error
java.lang.NoClassDefFoundError: Adder (wrong name: org/processing/Adder)
On the other hand, if I compile from src using
javac org/processing/Adder.java
and I can run it from src using java org.processing.Adder but STILL not from within the processing directory.
Q2) Does this mean that compilation is always relative to directory structure?
Finally, if I remove the package org.processing line from the start are the .class file I can compile and run from within the .class file's directory.
Q3) Why is all this the way it is? I can fully understand enforcing a directory structure for code development, but once you're in bytecode this seems a bit over the top, because now I can (apparently) only run the bytecode from one director (src) using java org.processing.Adder. Now, I'm sure I'm missing the point here, so if someone could point out what it is, that would be great.

The compiler has to be able to find related source code files when compiling. This is why the package and directory structure must agree for source code. Similarly, the JVM must be able to find referenced .class files. So the same directory structure is required at runtime. It's no more complex than that.

Q1) The issue here is that once you got into the folders that represent your package hierarchy, you set that as the working directory. It's gonna look inside of org/processing/Adder for the path org/processing/Adder (essentially looking from the root for org/processing/Adder/org/processing/Adder). You need to call it from the root with the full path. The purpose of packages is A: to organize related classes into groups. And B: Along with A, classes in package Foo.bar can't view private classes in other packages, as they are like internal classes for that package, only the package they're in can use them
Q2) Yes
Q3) The paths are used as a basic structure for the JVM to know where exactly the class files (each containing their bytecode) are. If you change where you call it from, your basically trying to change the location for the JVM to look for the class files, but their true location hasn't changed.

The short answer - Packages help keep your project structure well-organized, allow you to reuse names (try having two classes named Account), and are a general convention for very large projects. They're nothing more than folder structures, but why they're used can burn beginners pretty badly. Funnily enough, with a project less than 5 classes, you probably won't need it.
What, exactly is this line doing? Why is there, what's it's role.
The line
package org.processing
is telling Java that this class file lives in a folder called /org/processing. This allows you to have a class which is fully defined as org.processing.Processor here, and in another folder - let's say /org/account/processing, you can have a class that's fully defined as org.account.processing.Processor. Yes, both use the same name, but they won't collide - they're in different packages. If you do decide to use them in the same class, you would have to be explicit about which one you want to use, either through the use of either import statements or the fully qualified object name.
Does this mean that compilation is always relative to directory structure?
Yes. Java and most other languages have a concept known as a classpath. Anything on this classpath can be compiled and run, and by default, the current directory you're in is on the classpath for compilation and execution. To place other files on the classpath, you would have to use another command-line invocation to your compilation:
javac -sourcepath /path/to/source MainClass.java
...and this would compile everything in your source path to your current directory, neatly organized in the folder structure specified by your package statements.
To run them, as you've already established, you would need to include the compiled source in your classpath, and then execute via the fully qualified object name:
java -cp /path/to/source org.main.MainClass
Why is all this the way it is?
Like I said before, this is mostly useful for very large projects, or projects that involve a lot of other classes and demand structure/organization, such as Android. It does a few things:
It keeps source organized in an easy-to-locate structure. You don't have objects scattered all over the place.
It keeps the scope of your objects clear. If I had a package named org.music.db, then it's pretty clear that I'm messing with objects that deal with the database and persistence. If I had a package named org.music.gui, then it's clear that this package deals with the presentation side. This can help when you want to create a new feature, or update/refactor an existing one; you can remember what it does, but you can't recall its name exactly.
It allows you to have objects with the same name. There is more than one type of Map out there, and if you're using projects that pull that in, you'd want to be able to specify which Map you get - again, accomplished through either imports or the fully qualified object name.

For Q1: The package declaration allows you to guarantee that your class will never be mistaken for another class with the same name. This is why most programmers put their company's name in the package; it's unlikely that there will be a conflict.
For Q2: There is a one-to-one correspondence between the package structure and the directory structure. The short of it is that directories and packages must be the same, excepting the package is usually rooted under a folder called src.
For Q3: Once it's compiled, the class files will probably be in the appropriate folders in a jar file. Your ant or maven tasks will build the jar file so you won't really have to bother with it beyond getting the ant task set up the first time.

Public class outside a jar file containing multiple packages

So, I have a Java project containing several packages (like com.myapp.a , com.myapp.b, com.myapp.c) for better readability and I want to build a jar to use as a library in another project.
But I just want to expose only some classes and interfaces from this jar. The problem is that if I don't declare these classes public then they can't be seen inside the jar file between the packages (for example I have a class A in com.myapp.a package that is used in com.myapp.b package).
So how can I expose just what I want outside of the jar when I have multiple packages defined inside?

Currently Java does not address this problem directly.
OSGi adresses this problem by explicitly defining the exported package list.
Also hopefully this will be addressed with the Java 8 Modularity system as well.
So one option is to use OSGi, but this option does not work if the jar file is used directly rather than as an OSGi bundle.
Another option is to use code obfuscation (like Proguard), to obfuscate the packages you do not want to expose.

Eclipse "solved" this problem by making all classes available, but classes that were not intended to be used by clients were placed in packages whose name contains "internal". For example, that might mean that you have packages named "com.myapp.b" and "com.myapp.internal.b". It's made clear to users of the classes that internal classes are not guaranteed to be upwardly compatible or even present in later releases.

How does the java compiler find classes without header files?

When we refer to a class className in jar, how does it know whether it's defined or not when there's no header files(like in c/c++) ?

Java works with classloaders. Classes are needed for compilation, since it will perform static type checking to ensure that you are using the correct signatures of every method.
After compiling them, though, they are not linked like you have in a C/C++ compiler so basically every .class file is standalone. Of course this means that you will have to provide compiled classed used by your program when you are going to execute it. So it's a little bit different from how C and C++ prepare executables. You don't actually have a linking phase at all, it is not needed.
The classloader will dinamically load them by adding them to the runtime base used by the JVM.
Actually there are many classloaders that are used by the JVM that have different permissions and properties, you can also invoke it explicitly to ask for a class to be loaded. What happens can also be a sort of "lazy" loading in which the compiled .class code is loaded just when needed (and this loading process can throw a ClassNotFoundException if the asked class is not inside the classpath)

When you run the Java compiler or your application itself, you can specify a classpath which lists all the jars and directories you're loading classes from. A jar just contains a bunch of class files; these files have enough metadata in them that no extra header files are necessary.

The classes in the jar file contain all the required information (class names, method signatures etc) so header files are not needed.
When you compile multiple classes javac is clever enough to compile dependencies automatically so the system still works.

It looks at the classpath and tries to load the class from there to get its definition.

Java files are compiled into class files which are java bytecode. These class files reside in a file structure where the top level is pointed to by the classpath variable. Compiling in C/C++ creates object files which can be linked into executable binaries. Java only compiles into bytecode files which are pulled in by the JVM at runtime. The following provide more explanation.
http://en.wikipedia.org/wiki/Java_bytecode
http://en.wikipedia.org/wiki/Java_compiler
http://en.wikipedia.org/wiki/Java_Virtual_Machine

How is import done in Java?

For instance
import org.apache.nutch.plugin.Extension,
though used many times,
I've no much idea what is done essentially.
EDIT: Is org.apache.nutch.plugin essentially 4 directories or fewer than 4 like a directory named org.apache?

I think the question you might be trying to ask is, "What are packages in Java, and how does the import keyword relate to them?". Your confusion about directory structures might stem from the fact that some other languages have include directives that use file names to literally include the contents of the specified file in your source code at compile time. C/C++ are examples of languages that use this type of include directive. Java's import keyword does not work this way. As others have said, the import keyword is simply a shorthand way to reference one or more classes in a package. The real work is done by the Java Virtual Machine's class loader (details below).
Let's start with the definition of a "Java package", as described in the Wikipedia article:
A Java package is a mechanism for
organizing Java classes into
namespaces similar to the modules of
Modula. Java packages can be stored in
compressed files called JAR files,
allowing classes to download faster as
a group rather than one at a time.
Programmers also typically use
packages to organize classes belonging
to the same category or providing
similar functionality.
In Java, source code files for classes are in fact organized by directories, but the method by which the Java Virtual Machine (JVM) locates the classes is different from languages like C/C++.
Suppose in your source code you have a package named "com.foo.bar", and within that package you have a class named "MyClass". At compile time, the location of that class's source code in the file system must be {source}/com/foo/bar/MyClass.java, where {source} is the root of the source tree you are compiling.
One difference between Java and languages like C/C++ is the concept of a class loader. In fact, the concept of a class loader is a key part of the Java Virtual Machine's architecture. The job of the class loader is to locate and load any class files your program needs. The "primordial" or "default" Java class loader is usually provided by the JVM. It is a regular class of type ClassLoader, and contains a method called loadClass() with the following definition:
// Loads the class with the specified name.
// Example: loadClass("org.apache.nutch.plugin.Extension")
Class loadClass(String name)
This loadClass() method will attempt to locate the class file for the class with given name, and it produces a Class object which has a newInstance() method capable of instantiating the class.
Where does the class loader search for the class file? In the JVM's class path. The class path is simply a list of locations where class files can be found. These locations can be directories containing class files. It can even contain jar files, which can themselves contain even more class files. The default class loader is capable of looking inside these jar files to search for class files. As a side note, you could implement your own class loader to, for example, allow network locations (or any other location) to be searched for class files.
So, now we know that whether or not "com.foo.bar.MyClass" is in a class file in your own source tree or a class file inside a jar file somewhere in your class path, the class loader will find it for you, if it exists. If it does not exist, you will get a ClassNotFoundException.
And now to address the import keyword: I will reference the following example:
import com.foo.bar.MyClass;
...
public void someFunction() {
MyClass obj1 = new MyClass();
org.blah.MyClass obj2 = new org.blah.MyClass("some string argument");
}
The first line is simply a way to tell the compiler "Whenever you see a variable declared simply as type MyClass, assume I mean com.foo.bar.MyClass. That is what's happening in the case of obj1. In the case of obj2, you are explicitly telling the compiler "I don't want the class com.foo.bar.MyClass, I actually want org.blah.MyClass". So the import keyword is just a simple way of cutting down on the amount of typing programmers have to do in order to use other classes. All of the interesting stuff is done in the JVM's class loader.
For more information about exactly what the class loader does, I recommend reading an article called The Basics of Java Class Loaders

All it's doing is saving you typing. Instead of having to type "org.apache.nutch.plugin.Extension" every time you want to use it, the import allows you to refer to it by its short name, "Extension".
Don't be confused by the word "import" - it's not loading the .class file or anything like that. The class loader will search for it on the CLASSPATH and load it into perm space the first time your code requires it.
UPDATE: As a developer you have to know that packages are associated with directories. If you create a package "com.foo.bar.baz" in your .java file, it'll have to be stored in a directory com/foo/bar/baz.
But when you download a JAR file, like that Apache Nutch library, there are no directories involved from your point of view. The person who created the JAR had to zip up the proper directory structure, which you can see as the path to the .class file if you open the JAR using WinZip. You just have to put that JAR in the CLASSPATH for your app when you compile and run.

Imports are just hints to the compiler telling him how to figure out the full name of classes.
So if you have "import java.util.*;" and in your code you are doing something like "new ArrayList()", when the compiler processes this expression it first needs to find the fully qualified name of the type ArrayList. It does so by going thru the list of imports and appending ArrayList to each import. Specifically, when it appends ArrayList to java.util it get the FQN java.util.ArrayList. It then looks up this FQN in its class-path. If it finds a class with such a name then it knows that java.util.ArrayList is the correct name.

is "org.apache.nutch.plugin" essentially 4 directories?
If you have a class whose name is org.apache.nutch.plugin.Extension, then it is stored somewhere in the classpath as a file org/apache/nutch/plugin/Extension.class. So the root directory contains four nested subdirectories ("org", "apache", "nutch", "plugin") which in turn contain the class file.

import org.apache.nutch.plugin.Extension is a compilation time shortcut that allows you to refer to the Extension class without using the class' fully qualified name. It has no meaning at runtime, it's only a compilation time trick to save typing.
By convention the .class file for this class will be located in folder org/apache/nutch/plugin either in the file system or in a jar file, either of which need to be in your classpath, both at compile time and runtime. If the .class file is in a jar file then that jar file needs to be in your classpath. If the .class file is in a folder, then the folder that is the parent of folder "org" needs to be in your classpath. For example, if the class was located in folder c:\myproject\bin\org\apache\nutch\plugin then folder c:\myproject\bin would need to be part of the classpath.
If you're interested in finding out where the class was loaded from when you run your program, use the -verbose:class java command line option. It should tell you which folder or jar file the JVM found the class.

Basically when you make a class you can declare it to be part of a package. I personally don't have much experience with doing packages. However, afaik, that basically means that you are importing the Extension class from the org.apache.nutch.plugin package.

Buliding off of Thomas' answer, org.apache.nutch.plugin is a path to the class file(s) you want to import. I'm not sure about this particular package, but generally you'll have a .jar file that you add to your classpath, and your import statement points to the directory "./[classpath]/[jarfile]/org/apache/nutch/plugin"

you can't have a directory named org.apache as a package. the compiler won't understand that name and will look for the directory structure org/apache when you import any class from that package.
also, do not mistake the Java import statement with the C #include preprocessor instruction. the import statement is, like they've said, a shorthand for you to type fewer characters when referring to a class name.