We Keep Coding

Controlling Class Visibility in Java

My project has currently following package structure.
Here I have added a package name utils and defined all utility classes related to this module inside it. Those have been used by other packages (i.e. impl, internal), and because of that I have made classes and methods in util package public.
Because, it is public, not only classes in this module, classes in other modules can also access this and when I am coding using my IDE they are shown as coding suggestions.
I went through few research papers which describe how this can reduce the usability of the API and give a bad experience to developers who involve in the development [ref1, ref2].
I understand that java does not allow me to make classes inside util accessible to impl and internal packages and not to others.
Is it correct to put my utility classes to a package 'util'? Or should I put all classes that communicate with each other to the same package?

You are correct, something marked public becomes usable in any other package. In contrast to other languages, Java doesn't provide any control beyond that.
A simple workaround: it might be helpful to have one special package containing those public things that should be available to your external users.
Meaning: create something like com.whatever.product.api - and instruct your users that they are fine to use everything from there - but nothing else.
In other words: you make all those things public that you need to public; but you collect those things in a special place that you allow to be used by others.
It is worth mentioning though that Java9 will introduce the concept of modules, those allow you to define which of your packages should be public to users of your module. In that sense, java 9 modules allow you to distinguish between "internal" and "external" public.

Util classes are fine. Util classes are functionality that is used multiple places in a project but doesn't really belong to a specific class.
In a perfect world of OOP there wouldn't be any util classes, but it is however considered a good practice to create util classes if they do not belong to a specific class.
Your options for access modifiers are listed here:
https://docs.oracle.com/javase/tutorial/java/javaOO/accesscontrol.html
There is one way to achieve what you you want, it is however concidered a very bad practice. You can change your access modifiers of util classes to protected. This will make your util classes accessible from subclasses and packages. So if a class needs access to one of the util classes, then it has to extend this util class and thereby become a subclass. I cannot stress it enought, this is a very bad practice.

Java programming architecture - Interface and implemented class should be in same package?

I really wonder that put an Interface and a (Implemented) class in same package or separate is better. I usually put them in same package since I believe it is more convenient to compare.
But couple of days ago, I had an opportunity to use apache GenericObjectPool. Then I found such a package structure like org.apache.commons.pool.impl.
After all, my question is when should I use former one and when should I use the other. Thanks for your sincere answers in advanced:D

One of the tenets of composition is that interfaces are separated from their implementation.
Therefore it doesn't make much sense to constrain interfaces and their implementations to the same package.
(On a system I work on the interfaces are placed in a common area - loosely based on a concept of an interface repository).

If use interfaces just to lower the coupling, you can put interface and implementation into the same package. If you primarily count with more implementations if the interfaces, you should put them in separate package. This really depends on your design and I don't think there is a strict rule. Generally you should design you application as simple as possible and do the refactoring later. So when you start, you can put the implementation into the same package and only when it seems to be needed, you should split it.

If you define the responsibility of a package by its interfaces and classes, you attempt to abstract from concrete implementations. Hence, your package should consist of public types only.
If you continue to provide a default implementation for your package, I do not see any reason why not to put them into the same package. But I would always suggest to put your implementation into a separate project or more specific, to provide the API of your package and the implementation as two different artifacts (for example, jar-files).
Doing things this way, you support the following important aspects of abstraction:
replacement of one implementation with another
ensuring that client code depends only one the API and not the implementation (by adding only the API artifact to the clients classpath)

Java package structure

As a best-practice, is it ok for interfaces / classes in higher level packages to depend on interfaces / classes in lower level packages?
e.g.
can com.company.core.SomeClass depend on com.company.core.apackage.AnotherClass
Or should the dependencies flow the other way? I am trying to eliminate cycles between packages.

Typically, a user of a package might be interested in com.company.functionality.MainUse, which is implemented using com.company.functionality.implementationdetail.FiddleWithStuff, so I'd say you have to accept dependencies in that direction.
But the other direction is probably necessary as well, as implementation detail classes may need to implement interfaces that is part of the interface to the functionality.
So, unfortunatley, I think a strict directionality like that isn't a workable way to avoid cycles.

They usually flow the other way. com.mycompany.myproduct.Widget is an interface, and com.mycompany.myproduct.fancy.Button implements it.
There are however big exceptions to this general rule, and there is no reasons why it should be a rule.
Packages themselves are simply folders, except for primitive access rules on methods and fields.
What is far more important are dependencies between bundles, that is (in their basic form) jars. Having jar A depend on jar B depending on jar C depending again on jar A is a real problem. If you have the same circle between packages in the same jar, it can cause frustration in people who read or debug your code, or in elitists reading it, but is not a big issue.

Such dependencies are common even in the Java API itself.
Look at: http://download.oracle.com/javase/6/docs/api/java/awt/image/renderable/package-use.html
You can see that classes in java.awt use classes in java.awt.image.renderable.
(ADDENDUM) Bottom line: Both directions are used in practice. One is not, IMHO, necessarily better than the other.

Sure, you're using the so-called top-down approach, a package depends on its subpackages but not vice versa.
The bottom-up approach would be the other way around, a package would depend on its parent packages but not vice versa.
Either way is fine as long as you are able to maintain consistency, but the top-down approach is easier and more common.

Pros and cons of organizing the packages in the Java project

As projects that I'm working on grow larger and larger, I'm starting to be pretty unsure about dividing classes into packages. Suppose the project has a lot of layers and in these layers are interfaces and implementation and even more sublayers (components). I always end up with a lot of packages which starts to be little confusing.
So I want to know other people's approaches to working with packages. Do you prefer to have a lot of packages with few classes or few packages with a lot of classes? Do you prefer to separate implementations from interfaces? And so on... In general your daily habits of using packages and pros/cons of your approach.
Thank you for your posts.

Packages are meant to help you find things.
If they make it more confusing than it is, you're not doing something quite right. If the package structure isn't intuitive, it can actually be harder to find classes than in a flat structure.
There are two basic schools of organising classes into packages as far as I know:
Organising by module first. Here your higher level packages are different modules of the system and you might split it further by function.
Organising by function. Here you organise by function first (e.g. all controller classes in one package, all your data containers in another and so on) and optionally subdivide it by module.
There are pros and cons for both systems, I find them roughlty equal, although I prefer the module approach slightly.
The really important thing is though to follow one system and don't mix it with the other. Don't shoehorn classes into packages they don't belong to and don't be afraid to create a new package if your new class doesn't seem to belong to any of your existing ones.
If a package seems to have grown too large, you might want to split it. But the decision of whether a package should be split or not should be made on whether there is a clear conceptual divide between classes therein and not on numbers. A package with a single class is just as good as a package with thirty classes if it's obvious why they're there.
As for separating interfaces and implementations, first off, I'd probably question the need for them. Ever so often I come across interfaces with only one reasonable implementation, which makes me question their reason to exist. (Sometimes there is a good reason, but often there isn't.)
But if you have multiple implementations for a given interface, then yes, I'd separate them. The interface would be com.foo.Bar and the implementations would be something like com.foo.bars.CrowBar, com.foo.bars.TaskBar for example. Obviously, if your implementations belong to different modules, then you would change it to com.foo.moduleX.bars.CrowBar or com.foo.bars.moduleX.CrowBar, depending on which system you're following.
Re-reading all this, it does sound kind of complicated, but probably the first sentence is the most important: don't follow packaging principles blindly, packages should help you find classes, not hinder you.

I prefer to limit the scope of my classes and methods to private or package protected wherever possible (so they don't clutter Eclipse's autocomplete:) which automatically means that packages can contain quite a few classes.
I do prefer to separate business data objects (DAO's) from their repositories/retrieval services, which are separated from business logic objects and views.
Sets of related functionality with no cross dependencies tend to go in their own artifact as I tend to reuse logic. Especially handy when you start playing with OSGi and independent services.
One thing is important, the publicly exported interface (public interfaces, enums and classes) need to be somewhat related so the package documentation shows some cohesion.

Is having a fine-grained package structure a good or a bad thing?

I recently looked at a Java application which had a very fine-grained package structure. Many packages only contained one or two classes and many sub packages. Also many packages contained more sub packages than actual classes.
Is this a good or a bad thing?

IMO, it is a bad thing, though not a real show-stopper in terms of maintainability.
The disadvantages are that it makes classes harder to find, and that it makes the package names more verbose. The former applies more when you are not using an IDE.
It could be argued that it helps modularization in conjunction with "package private" scoping. But conversely, you could also argue that over-packagization actually does the opposite; i.e. forcing you to use public where you wouldn't have had to if you'd been less fine-grained / pedantic.

The actual number of types that end up in one particular package is not that important. It is how you arrive at your package structure.
Things like abstraction ("what" instead of "how", essentially "public" API), coupling (the degree of how dependent a package is on other packages) and cohesion (how interrelated is the functionality in one package) are more important.
Some guidelines for package design (mostly from Uncle Bob) are for example:
Packages should form reusable and releasable modules
Packages should be focussed for good reusability
Avoid cyclic dependencies between packages
Packages should depend only on packages that change less often
The abstraction of a package should be in proportion to how often it changes
Do not try to flesh out an entire package structure from scratch (You Are Not Going To Need It). Instead let it evolve and refactor often. Look at the import section of your Java sources for inspiration on moving types around. Don't be distracted by packages that contain only one or a few types.

I think finer grained package structure is a good thing. The main reason is it can help minimize your dependencies. But be careful... if you break things up too much that actually belong together, you will end up with circular dependencies!
As a rule of thumb, I usually have an interface (or group of related interfaces) in a package. And in subpackages I will have implementations of those interfaces (instead of having all the implementations in the same package). That way a client can just depend on the interface and implementation of interest... and not all the other stuff they don't need.
Hope this helps.

It is subjective, of course, but I generally prefer to decide my packages in a way they would contain at least 3-4 classes, but not more than about 13-15. It makes understanding better, while not cluttering the project. For others it might be different, though.
In case a package grows more than 13-15 classes, a subpackage is asking to emerge.

A package is a unit of encapsulation.
Ideally, it exposes a public API via interface(s) and hides implementation detail in package private classes.
The size of the package is therefore the number of classes needed to implement the public API.

There is also the magical number 7 +/- 2. Within physiological circles it has been shown that this is a basic limit on our ability to comprehend a given set of things. How much we can keep in short term memory and process at the same time before our brains need to start swapping. It is not a bad rule of thumb I've found when coding, ie once a package starts getting bigger than 10 classes it is time to think seriously about splitting it up, but below 5 packages it is really not worth it. Also applies well to things like menu organisation. See Millers paper, or google.

When writing from scratch my way is to begin all classes in the root package (com.example.app1). When there is a certain amount of interrelated classes that "do a whole thing" is time to create a package. After some time developing helper classes may go to a generic package (ex. com.example.app1.misc, com.example.app1.utils) to unload the root package.
So i try to keep clean the root package.
Fine-grain is not bad but as other said often refactoring produce a compact package structure.