We Keep Coding

Memory leak not unloading Singletons when bringing down a Java EE application

This is a high-voted example of memory leak from StackOverflow:
Not unloading Singletons when bringing down a Java EE application. Apparently, the Classloader that loaded the singleton class will retain a reference to the class, and hence the singleton instance will never be collected. When a new instance of the application is deployed, a new class loader is usually created, and the former class loader will continue to exist due to the singleton.
1) I do not understand how can I "unload" a singleton and what is meant by "bringing down a Java EE application". Could you provide code examples (wrong and right code samples) and scenarios?
2) From the same stackoverflow post:
Take any web application running in any servlet container (Tomcat, Jetty, Glassfish, whatever...). Redeploy the app 10 or 20 times in a row (it may be enough to simply touch the WAR in the server's autodeploy directory.
Unless anybody has actually tested this, chances are high that you'll
get an OutOfMemoryError after a couple of redeployments, because the
application did not take care to clean up after itself. You may even
find a bug in your server with this test.
The problem is, the lifetime of the container is longer than the
lifetime of your application. You have to make sure that all
references the container might have to objects or classes of your
application can be garbage collected.
If there is just one reference surviving the undeployment of your web app, the corresponding classloader and by consequence all
classes of your web app cannot be garbage collected.
Threads started by your application, ThreadLocal variables, logging
appenders are some of the usual suspects to cause classloader leaks.
I cannot understand how is it possible that container (Tomcat classes/objects) holds references to objects or classes of my application and it is my fault. Automatic memory management means I don't have to care about freeing memory, right? So what shall I take care of if my application runs within Tomcat or another container?

A single jvm can be used as "application server". It hosts containers, coming in as "packages" (EAR or WAR files for example) that can be dynamically added / removed from the jvm.
You achieve that by using the capabilities of the class loader. But a class loader keeps track of all the classes it loaded. So, to free up the old class loader it has to forget all classes it knows about. But it can't for singletons done wrong.
Or quoting from IBM:
An object retains a reference to the class it is an instance of. A class retains a reference to the class loader that loaded it. The class loader retains a reference to every class it loaded. Retaining a reference to a single object from a web application pins every class loaded by the web application. These references often remain after a web application reload. With each reload, more classes are pinned which leads to an out of memory error
By request from the OP, I tried to find examples for "bad" respectively "working" singleton implementations, but I couldn't find any.
But to give a different perspective: since we have enums, we can use enums to implement a singleton (see here). So, probably the reasonable answer today is: simply use enums. And given the fact that there isn't much to find on the internet regarding this subject, my (personal) gut feeling is: this is simply not a relevant problem in the real world (any more).

Tomcat: how does Tomcat unloads classes internally?

I am looking for a way to replace a jar with old functionality with a new one without stopping the whole application. I have googled for this a little bit (Unloading classes in java?, Dynamically loadable and unloadable application modules in Java - how?), and have found 3 possible solutions:
OSGI, which looks too complicated for my purposes: class unloading is the only thing I need from this powerful specification.
Own implementation, which implies creating own Classloader, loading jars with its help, and when time comes to dismiss the jar - expose Classloader instance to GC and hope that GC will remove it together with all loaded jars, which may not happen immediately.
Copy the stuff application servers do :) That is the purpose of my question.
Is there any explanation on how Tomcat unloads the classes (wars)? Is it possible to reuse this in my application?
PS
Just while I was writing this post 2 strange ideas came into my head:
may be we can start another instance of application with new set of jars and switch input and output data flows from old one to new one. If there is a place to preserve state - it can be more quick than full restart.
may be Spring may help? It can dynamically register beans, but this looks like somewhat ugly solution, and it cannot unload them anyway; may be, Spring Boot may help?

A class can be unloaded only when it and its class loader are unreachable and thus eligible for garbage collection. Since the class loader can reach every class it loaded and every class loaded by a class loader can reach its class loader and every object which is an instance of a class can reach its class, this means that the class loader, all the classes it loaded and all the objects which are an instance of those classes must all be unreachable and eligible to be garbage collected. Only then, can a class be "unloaded" when that complete object graph is garbage collected.

I can only partially answer you questions, however better than nothing... You are right, modularity is a quite a tedious topic (at least until there's Java 9).
Ad idea 2 (Spring): I already spent some thoughts on Spring Boot over here and came to the conclusion that I'll stick either to OSGI (it's worth getting into it) or to a pure Microservice architecture.

I suppose that you are not doing whole redeployment of application, but don't want to restart whole application every time while you are developing it
If you want to change functionality of methods without changing class or method signatures you can use Spring Loaded
Add dependency to your project https://mvnrepository.com/artifact/org.springframework/springloaded/1.2.6.RELEASE
Set JVM option -javaagent:path\springloaded-version.jar -noverify
This option can be set in several ways depending on how you run tomcat
I have Broadleaf eccomerce web site and I added this option to MAVEN_OPTS in batch script that runs embedded tomcat or you can configure that in Run configuration for Tomcat VM options
read http://docs.spring.io/spring-boot/docs/current/reference/html/howto-hotswapping.html

How does JVM deal with dynamic classes

Class definitions are stored in the Method Area, as the Java Virtual Machine Specification says (The Java® Virtual Machine Specification Java SE 7 Edition):
The method area is created on virtual machine start-up. Although the
method area is logically part of the heap, simple implementations may
choose not to either garbage collect or compact it.
As we know, some bytecode tools like ASM, cglib, javassist, Hibernate and Spring frameworks are using them. For a common class file, JVM loads and parses and initializes and finally uses it, I am confused about how does JVM deal with the classes generated by bytecode tools dynamically. My questions are:
If JVM loads, parses and initializes the dynamic classes as the common class file?
Are they stored in the Method Area as well?
How does JVM unload and clean the dynamic class definitions to prevent itself from occurring an OutOfMemoryError?

All class are loaded at runtime, possibly compiled to native code. As such there is nothing special about class generated after the program has started.
If JVM loads, parses and initializes the dynamic classes as the common class file?
It loads the same way as class which existed when the program started.
Are they stored in the Method Area as well?
They are stored the same way in fact it is hard to tell if a class is dynamic or not.
How does JVM unload and clean the dynamic class definitions to prevent itself from occurring an OutOfMemoryError?
The JVM can unload classes when the ClassLoader they are in is unloaded. This is true whether the classes are dynamic or not.
how could the JVM know to treat dynamic classes any differently than 'normal' classes?
There is one example of dynamic classes which are special. These are the lambda classes which are generated at runtime. What makes them different is they are not bound to a class loader, they don't even have a normal class name. They get unloaded when all the instances of that class are unused.
From InnerClassLambdaMetafactory
UNSAFE.defineAnonymousClass(targetClass, classBytes, null);
The class has no class name attached to it (nor a ClassLoader)

There is no difference between .class data on the file system, inside a jar file or dynamically created by tools like ASM.
In fact when the class loader loads the class, it's just a byte[] how that byte array is populated with .class data is up to the class loader to get it from the filesystem, inside a jar or zip file, from a URL, dynamically generated etc.
If JVM loads, parses and initializes the dynamic classes as the common class file?
As I said it is exactly the same. The JVM doesn't know the difference.
Are they stored in the Method Area as well?
Again, JVM doesn't know the difference so I guess.
How does JVM unload and clean the dynamic class definitions to prevent itself from occurring an OutOfMemoryError
The same way as non-dynamic classes. by unloading although I don't know how often this happens

Java, runtime class reloading

I am looking for a way to reload a class into Java at runtime. The motivation is to make debugging more efficient. The application is a typical client/server design that synchronously processes requests. A "handler" object is instantiated for each request. This is the only class I intend to dynamically replace. Since each request deals with a fresh instance, reloading this class won't have any side-effects. In short, I do not want to restart the entire application every time there is a change to this module.
In my design, the Java process becomes aware that a .class file has been updated in the classpath in between requests. When this happens, the "handler" class is unloaded and a new one is loaded.
I know I can use the classLoader interface to load in a new class. I seem to be having trouble finding the proper way of "unloading".

Classes will be unloaded and garbage collected like any other object, if there is no remaining reference to them. That means there must be no reachable instance of the class (as loaded by that particular classloader instance) and the classloader instance itself must be eligible for garbage collection as well.
So basically, all you have to do is to create a new classloader instance to load the new version of the class, and make sure that no references to instances of the old version remain.

I believe that you actually need to have a hierarchy of classloaders, and in order to reload you actually get rid of the low level classloader (by normall GC means), and hence all the classes it loaded. So far as I know this technique is used by Java EE app servers for reloading applications, and there's all manner of fun results when framework code loaded in one classloader wants to use classes loaded somewhere else.

As of 2015 also java's class reloading is a missing feature.
Use OSGi to create a class reloading application.
Use jrebel for testing. There are a few others which does the same thing.
Use application server and externalize the parts which you want to reload into a separate web application. Then keep deploying/undeploying. You will eventually get some perm gen space overflow kind of errors due to dangling old ClassLoader instances.
Use a script runner to execute parts of changeable code. JSR-223 Java Scripting API support for the scripting language "Java".
I had written a series about class reloading. But all of those methods are not good for production.
The blog and source codes in google sources
IMHO this class reloading is messy in java and its not worth trying it. But I would very much like this to be a specification in java.

Can I dynamically unload and reload (other versions of the same) JAR?

I am writing a server program which is used to run unit tests of an API
(displaying lots of information and providing web access to control
/ monitor the whole thing)...
This API is known to the server during compile time and is provided
as a JAR.
To be able to compare between unit test results of different versions
of the API (without restarting the server),
I want to be able to unload the 'current' version of the API,
and to reload a newer one (or an older one).
I don't want to use URLClassLoader and invoke every single
method by name
( using getDeclaredMethod("someMethod") ),
because the server heavily depends on the API and it would be
complicated to 'wrap' every method call in such dirty way.
I was thinking: Since all interfaces of all versions of the JAR
are same, couldn't I do it by somehow reloading an other version
of the JAR (without that by-name-invokation?).
Note: I am using latest Java SE (6) and Java EE (5).
If you think, what I'm trying to achieve is not possible,
please suggest a 'workaround' or a different concept.

I think if you load a class using
Class.forName(clsname, init, classloader);
(Javadoc here) you will get an instance of the class provided by the given classloader. Everything loaded because of that class will also be loaded via the same classloader.
As long as you're very careful with the objects instantiated from this point on (to allow for GC), you should be able to reload different versions. I did this once before with Java 1.3, it took a lot of debugging, but at the end I had a "bootstrap" application that loaded a Runnable class by name and was able to "soft-restart" by instantiating a new classloader against a different URL and going again.

You can use the opensource package : JclLoader which helps in loading different versions of the same jar. This was also a need in one of our systems to do testing .
Link: http://sourceforge.net/projects/jcloader/

You could programatically modify your classpath to reflect your JAR changes.
Here is how I would do it:
URLClassLoader urlClassLoader = (URLClassLoader) ClassLoader.getSystemClassLoader();
Method m = URLClassLoader.class.getDeclaredMethod("addURL", new Class[]{URL.class});
m.setAccessible(true);
m.invoke(urlClassLoader, jarFile.toURI().toURL());
String cp = System.getProperty("java.class.path");
if (cp != null) {
cp += File.pathSeparatorChar + jarFile.getCanonicalPath();
} else {
cp = jarFile.toURI().getPath();
}
System.setProperty("java.class.path", cp);
where jarFile is the version of the jar you want to use/overwrite.

OSGi is a framework that will allow you to do it. JSR 277 the Java Module System is designed for doing that as well (I think). I have not followed the OSGi -vs- JSR 277 debate, so I don't know f they are trying to marge them at all.
You can roll your own with class loaders, but it'll be less "fun".

Yes. I've seen it done at a NFJS conference. It's how things like web containers support hot deployment of applications and involves taking advantage of the scope of class loaders. In order to accomplish it you need to create a new class loader and use that to load the library in question.. then throw the loader away (or not) and create another when you want to reload. You may also have to override the behaviour of the class loader (I remember something about class loaders getting classes via their parent first by default.) Also, I remember a warning that objects created by different class loaders are not compatible (not of the same type) with each other even if the .class file is exactly the same.
It's mostly deep magic to me though. ;-)

Probably not. The Java classloader doesn't really support run-time loading; even the available classloaders are hacks that use a proxy object.