We Keep Coding

StringCoding has threadLocal [duplicate]

Does any one have an example how to do this? Are they handled by the garbage collector? I'm using Tomcat 6.

The javadoc says this:
"Each thread holds an implicit reference to its copy of a thread-local variable as long as the thread is alive and the ThreadLocal instance is accessible; after a thread goes away, all of its copies of thread-local instances are subject to garbage collection (unless other references to these copies exist).
If your application or (if you are talking about request threads) container uses a thread pool that means that threads don't die. If necessary, you would need to deal with the thread locals yourself. The only clean way to do this is to call the ThreadLocal.remove() method.
There are two reasons you might want to clean up thread locals for threads in a thread pool:
to prevent memory (or hypothetically resource) leaks, or
to prevent accidental leakage of information from one request to another via thread locals.
Thread local memory leaks should not normally be a major issue with bounded thread pools since any thread locals are likely to get overwritten eventually; i.e. when the thread is reused. However, if you make the mistake of creating a new ThreadLocal instances over and over again (instead of using a static variable to hold a singleton instance), the thread local values won't get overwritten, and will accumulate in each thread's threadlocals map. This could result in a serious leak.
Assuming that you are talking about thread locals that are created / used during a webapp's processing of an HTTP request, then one way to avoid the thread local leaks is to register a ServletRequestListener with your webapp's ServletContext and implement the listener's requestDestroyed method to cleanup the thread locals for the current thread.
Note that in this context you also need to consider the possibility of information leaking from one request to another.

Here is some code to clean all thread local variables from the current thread when you do not have a reference to the actual thread local variable. You can also generalize it to cleanup thread local variables for other threads:
private void cleanThreadLocals() {
try {
// Get a reference to the thread locals table of the current thread
Thread thread = Thread.currentThread();
Field threadLocalsField = Thread.class.getDeclaredField("threadLocals");
threadLocalsField.setAccessible(true);
Object threadLocalTable = threadLocalsField.get(thread);
// Get a reference to the array holding the thread local variables inside the
// ThreadLocalMap of the current thread
Class threadLocalMapClass = Class.forName("java.lang.ThreadLocal$ThreadLocalMap");
Field tableField = threadLocalMapClass.getDeclaredField("table");
tableField.setAccessible(true);
Object table = tableField.get(threadLocalTable);
// The key to the ThreadLocalMap is a WeakReference object. The referent field of this object
// is a reference to the actual ThreadLocal variable
Field referentField = Reference.class.getDeclaredField("referent");
referentField.setAccessible(true);
for (int i=0; i < Array.getLength(table); i++) {
// Each entry in the table array of ThreadLocalMap is an Entry object
// representing the thread local reference and its value
Object entry = Array.get(table, i);
if (entry != null) {
// Get a reference to the thread local object and remove it from the table
ThreadLocal threadLocal = (ThreadLocal)referentField.get(entry);
threadLocal.remove();
}
}
} catch(Exception e) {
// We will tolerate an exception here and just log it
throw new IllegalStateException(e);
}
}

There is no way to cleanup ThreadLocal values except from within the thread that put them in there in the first place (or when the thread is garbage collected - not the case with worker threads). This means you should take care to clean up your ThreadLocal's when a servlet request is finished (or before transferring AsyncContext to another thread in Servlet 3), because after that point you may never get a chance to enter that specific worker thread, and hence, will leak memory in situations when your web app is undeployed while the server is not restarted.
A good place to do such cleanup is ServletRequestListener.requestDestroyed().
If you use Spring, all the necessary wiring is already in place, you can simply put stuff in your request scope without worrying about cleaning them up (that happens automatically):
RequestContextHolder.getRequestAttributes().setAttribute("myAttr", myAttr, RequestAttributes.SCOPE_REQUEST);
. . .
RequestContextHolder.getRequestAttributes().getAttribute("myAttr", RequestAttributes.SCOPE_REQUEST);

Reading again the Javadoc documentation carefully:
'Each thread holds an implicit reference to its copy of a thread-local variable as long as the thread is alive and the ThreadLocal instance is accessible; after a thread goes away, all of its copies of thread-local instances are subject to garbage collection (unless other references to these copies exist).
'
There is no need to clean anything, there is an 'AND' condition for the leak to survive. So even in a web container where thread survive to the application,
as long as the webapp class is unloaded ( only beeing reference in a static class loaded in the parent class loader would prevent this and this has nothing to do with ThreadLocal but general issues with shared jars with static data ) then the second leg of the AND condition is not met anymore so the thread local copy is eligible for garbage collection.
Thread local can't be the cause of memory leaks, as far the implementation meets the documentation.

I would like to contribute my answer to this question even though it's old. I had been plagued by the same problem (gson threadlocal not getting removed from the request thread), and had even gotten comfortable restarting the server anytime it ran out of memory (which sucks big time!!).
In the context of a java web app that is set to dev mode (in that the server is set to bounce every time it senses a change in the code, and possibly also running in debug mode), I quickly learned that threadlocals can be awesome and sometime be a pain. I was using a threadlocal Invocation for every request. Inside the Invocation. I'd sometimes also use gson to generate my response. I would wrap the Invocation inside a 'try' block in the filter, and destroy it inside a 'finally' block.
What I observed (I have not metrics to back this up for now) is that if I made changes to several files and the server was constantly bouncing in between my changes, I'd get impatient and restart the server (tomcat to be precise) from the IDE. Most likely than not, I'd end up with an 'Out of memory' exception.
How I got around this was to include a ServletRequestListener implementation in my app, and my problem vanished. I think what was happening is that in the middle of a request, if the server would bounce several times, my threadlocals were not getting cleared up (gson included) so I'd get this warning about the threadlocals and two or three warning later, the server would crash. With the ServletResponseListener explicitly closing my threadlocals, the gson problem vanished.
I hope this makes sense and gives you an idea of how to overcome threadlocal issues. Always close them around their point of usage. In the ServletRequestListener, test each threadlocal wrapper, and if it still has a valid reference to some object, destroy it at that point.
I should also point out that make it a habit to wrap a threadlocal as a static variable inside a class. That way you can be guaranteed that by destroying it in the ServeltRequestListener, you won't have to worry about other instances of the same class hanging around.

#lyaffe's answer is the best possible for Java 6. There are a few issues that this answer resolves using what is available in Java 8.
#lyaffe's answer was written for Java 6 before MethodHandle became available. It suffers from performance penalties due to reflection. If used as below, MethodHandle provides zero overhead access to fields and methods.
#lyaffe's answer also goes through the ThreadLocalMap.table explicitly and is prone to bugs. There is a method ThreadLocalMap.expungeStaleEntries() now available that does the same thing.
The code below has 3 initialization methods to minimize the cost of invoking expungeStaleEntries().
private static final MethodHandle s_getThreadLocals = initThreadLocals();
private static final MethodHandle s_expungeStaleEntries = initExpungeStaleEntries();
private static final ThreadLocal<Object> s_threadLocals = ThreadLocal.withInitial(() -> getThreadLocals());
public static void expungeThreadLocalMap()
{
Object threadLocals;
threadLocals = s_threadLocals.get();
try
{
s_expungeStaleEntries.invoke(threadLocals);
}
catch (Throwable e)
{
throw new IllegalStateException(e);
}
}
private static Object getThreadLocals()
{
ThreadLocal<Object> local;
Object result;
Thread thread;
local = new ThreadLocal<>();
local.set(local); // Force ThreadLocal to initialize Thread.threadLocals
thread = Thread.currentThread();
try
{
result = s_getThreadLocals.invoke(thread);
}
catch (Throwable e)
{
throw new IllegalStateException(e);
}
return(result);
}
private static MethodHandle initThreadLocals()
{
MethodHandle result;
Field field;
try
{
field = Thread.class.getDeclaredField("threadLocals");
field.setAccessible(true);
result = MethodHandles.
lookup().
unreflectGetter(field);
result = Preconditions.verifyNotNull(result, "result is null");
}
catch (NoSuchFieldException | SecurityException | IllegalAccessException e)
{
throw new ExceptionInInitializerError(e);
}
return(result);
}
private static MethodHandle initExpungeStaleEntries()
{
MethodHandle result;
Class<?> clazz;
Method method;
Object threadLocals;
threadLocals = getThreadLocals();
clazz = threadLocals.getClass();
try
{
method = clazz.getDeclaredMethod("expungeStaleEntries");
method.setAccessible(true);
result = MethodHandles.
lookup().
unreflect(method);
}
catch (NoSuchMethodException | SecurityException | IllegalAccessException e)
{
throw new ExceptionInInitializerError(e);
}
return(result);
}

The JVM would automatically clean-up all the reference-less objects that are within the ThreadLocal object.
Another way to clean up those objects (say for example, these objects could be all the thread unsafe objects that exist around) is to put them inside some Object Holder class, which basically holds it and you can override the finalize method to clean the object that reside within it. Again it depends on the Garbage Collector and its policies, when it would invoke the finalize method.
Here is a code sample:
public class MyObjectHolder {
private MyObject myObject;
public MyObjectHolder(MyObject myObj) {
myObject = myObj;
}
public MyObject getMyObject() {
return myObject;
}
protected void finalize() throws Throwable {
myObject.cleanItUp();
}
}
public class SomeOtherClass {
static ThreadLocal<MyObjectHolder> threadLocal = new ThreadLocal<MyObjectHolder>();
.
.
.
}

final ThreadLocal<T> old = backend;
// try to clean by reflect
try {
// BGN copy from apache ThreadUtils#getAllThreads
ThreadGroup systemGroup = Thread.currentThread().getThreadGroup();
while (systemGroup.getParent() != null) {
systemGroup = systemGroup.getParent();
}
int count = systemGroup.activeCount();
Thread[] threads;
do {
threads = new Thread[count + (count / 2) + 1]; //slightly grow the array size
count = systemGroup.enumerate(threads, true);
//return value of enumerate() must be strictly less than the array size according to javadoc
} while (count >= threads.length);
// END
// remove by reflect
final Field threadLocalsField = Thread.class.getDeclaredField("threadLocals");
threadLocalsField.setAccessible(true);
Class<?> threadLocalMapClass = Class.forName("java.lang.ThreadLocal$ThreadLocalMap");
Method removeMethod = threadLocalMapClass.getDeclaredMethod("remove", ThreadLocal.class);
removeMethod.setAccessible(true);
for (int i = 0; i < count; i++) {
final Object threadLocalMap = threadLocalsField.get(threads[i]);
if (threadLocalMap != null) {
removeMethod.invoke(threadLocalMap, old);
}
}
}
catch (Exception e) {
throw new ThreadLocalAttention(e);
}

Local variables still cause memory leak when the thread wait()

I have met a memory leak in the following code:
#Override
public void run() {
while (true) {
if(!isRunning) {
break;
}
ThreadHandler handler = threadHandlerRef.get();
if(handler == null) {
break;
}
handler.sendMessage(Message.obtain(handler, 1, this));
synchronized (this) {
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
The memory leak is like below:
In com.****.****:4.2.10:6796.
* com.****.****.SmartConfigMainActivity has leaked:
* GC ROOT com.****.****..NetworkDetector$DThread.<Java Local>
* references android.os.Message.obj
* references com.****.****.$3.a
* references com.****.****.mCommonBindView
* references com.****.****.CommonBindView.mContext
* leaks com.****.****. instance
The thread, as the gc root, holding the local varaiables such as Message. So, the wait() call, from the link:
tells the currently executing thread go to sleep (not use any cpu).
releases the lock so other threads can wake up and take the lock.
Seems wait() also result in holding the local variables. I did not realize this before. Where these local variables? Is it still in the "VM Stack"? Is there more tutorial about this?
=================UPDATE=======================
The SmartConfigMainActivity does not have any relations with the NetworkDetector. I think the Message is re-used in the Looper, which cuased this memory-leak. But I cannot control this re-use behaviour of android.
As I see, jvm should save the local variable to prevent it from gc because when wait() returns, it should come back to the origin state and use these variables. But I did not see any tutorial about this

Old Generation memory increases when using ThreadLocal

I have class Collector and ThreadLocalScope like this:
Collector {
Collector() {
events = new LinkedList<>();
}
add(Event e) {
events.add(e);
}
flush() {
LinkedList<Event> copy = events;
new Thread(() -> {
for (Event e : copy) {
sendToServer(e);
}
copy.clear();
).start();
events = new LinkedList<>();
}
}
ThreadLocalScope {
public static ThreadLocal<Collector> local = new ThreadLocal<>() {
protected Collector initialValue() {
return new Collector();
}
}
}
Collector simply adds events and when flush is called sends those events to an API in a new thread. The Collector is initialized in a ThreadLocal.
I also have a Job class which is executed several times (using Quartz). When defined like this everything works great:
Job {
execute() {
for (int i = 0; i < 100,000; i++) {
ThreadLocalScope.get().add(new Event());
}
ThreadLocalScope.get().flush();
}
}
However if instead I hold onto Collector like this:
Job {
Collector collector;
Job() {
collector = ThreadLocalScope.get();
}
execute() {
for (int i = 0; i < 100,000; i++) {
collector.add(new Event());
}
collector.flush();
}
}
I see my Old Generation Memory usage increasing rapidly and Stop-the-world Garbage Collection cycles happening all the time. The only difference is I have added Collector as a member variable rather than calling ThreadLocalScope.get() every time.
The increase could only mean that the Events are being moved into Old Generation. But why would that happen? Collector immediately clears all its references to the Events, so even if it is not GCed, the events should be.

I said:
I think you might have a thread-safety issue here.
Incorrect. I think it is simpler than that.
In the first version you are calling ThreadLocalScope.get() in the context of the thread that is executing the job.
In the second version you are calling ThreadLocalScope.get() in the contrext of the thread that is creating the Job() object. It is then squirreled away in a variable and used later in the executor thread. Assuming that the Job() objects are all created on the same thread, that means that your execute() methods share the same Collector object. And they are potentially running on different threads. Since Collector is not thread-safe, that is a hazard.
There is another thing that you might not be aware of. It is likely that Quartz is using a thread pool. That means that when an execute() call terminates, the thread goes back to the pool. Next time around, if Quartz uses the same thread, it will reuse the Collector object from last time.

can java 8 lambdas cause memory leaks?

well I found this code in a blog, and wanted to understand why it would cause a memory leak, if it is potential of causing a memory leak.
class Test {
public static void main(String[] args) {
Runnable runnable = new EnterpriseBean()
.runnable();
runnable.run(); // Breakpoint here
}
}
#ImportantDeclaration
#NoMoreXML({
#CoolNewValidationStuff("Annotations"),
#CoolNewValidationStuff("Rock")
})
class EnterpriseBean {
Object[] enterpriseStateObject =
new Object[100_000_000];
Runnable runnable() {
return () -> {
System.out.println("Hello from: " + this);
};
}
}

The provided code does not have a memory leak, and the blog entry from which it is drawn does not say otherwise. What it says is that the object returned by EnterpriseBean.runnable() has much (much) larger state than you might naively expect, and that that state cannot be garbage collected before the Runnable itself is.
However, there is nothing in that code that would prevent the Runnable from eventually being collected, and at that time all the extra state will be eligible for collection, too.
So no, the code is not an example of a memory leak, and does not suggest a means to produce one.

Is synchronized needed here

I have a java applet. A class inside that applet is creating a thread to do some work, waiting 30 seconds for that work to complete, if its not completed in 30 secs it sets a Boolean to stop the thread. The wait and Boolean change are in a synchronized block, Is this necessary considering there is no other thread running aside from these 2.
System.out.println("Begin Start Session");
_sessionThread = new SessionThread();
_sessionThread.start();
synchronized (_sessionThread)
{
_sessionThread.wait(30000);
_sessionThread._stopStartSession = true;
}
Why couldn't I just do this instead.
System.out.println("Begin Start Session");
_sessionThread = new SessionThread();
_sessionThread.start();
_sessionThread.wait(30000);
_sessionThread._stopStartSession = true;
SessionThread run method. Invokes a JNI method to call a dll to open a program window.
public void run()
{
try
{
startExtraSession();
}
catch (Throwable t)
{
t.printStackTrace();
}
notify();
}
private native void openSessionWindow(String session_file);
private void startExtraSession()
{
final String method_name = "startExtraSession";
String title = _sessionInfo._title;
long hwnd = 0;
openSessionWindow(_sessionInfo._configFile);
try
{
//Look for a window with the predefined title name...
while ((hwnd = nativeFindWindow(title)) == 0 && !_stopStartSession)
{
Thread.sleep(500);
}
}
catch(Throwable t)
{
t.printStackTrace();
}
}
1. Is the synchronized really needed?
2. Is there a better way to accomplish this aside from using threads?

A given thread is required to own a lock on a object to be able to call wait(long) on it. This is achieved by using a synchronized block on the said object.
See J2SE specification on using wait.
Acquiring a lock/monitor in java can be done in various ways:
In a synchronized (non-static) method, the thread owns a monitor on the object referenced by this.
In a static synchronized method, the thread owns a monitor on the Class<?> descriptor for the class that defines the said method.
In a synchronized(x) block, the thread owns a monitor on x.
That lock will be released if:
You get outside of the synchronized code block (be it a method, static method, or explicit block).
You have called wait() or one of its variations (and you'll re-acquire it just before the method returns).
Both these two lists may omit specific cases but should cover at least a large portion of the typical use cases.

There's a very simple reason that you need synchronized to call wait
The synchronized makes sure that nobody is calling notify or notifyAll at the same time you're calling wait
For example: Thread 1
synchronized( obj )
{
triggerActionOnThread2();
obj.wait();
}
Thread 2 (triggered by triggerActionOnThread2)
...
synchronized( obj )
{
obj.notify();
}
If you don't have the synchronized blocks, then the notify might happen before (or during) the wait, and then the wait misses the notify, and you can hang Thread 1.
Imagine the above blocks of code without the synchronized blocks, and imagine if Thread 2 is executed all the way through the notify before the wait gets called.
BTW, I ask this very question on interviews for Java engineers when the job will involve multithreaded programming.

Can you please post SessionThread code? You cannot wait if you don't own the lock, so you need synchronized (_sessionThread) to do _sessionThread.wait(30000); Not sure what's with _sessionThread._stopStartSession = true;

If the boolean is the only shared state between the threads, declaring the boolean transient will guarantee that changes to it are seen between the threads as would a synchronization block around access to the boolean.