I am working on a module where a one-time cache is loaded using the static workflow. Cache loading usually takes around an hour. To improve the performance I am thinking about running these tasks in parallel using a thread pool. Here is the sample code.
Application Startup class:
public class AppStart {
public static void main(String[] args) {
Cache.isValid(); // this will trigger the static workflow
// ...
}
}
Cache loader class:
public class Cache {
static {
System.out.println("Static block initialization started!");
initialize();
System.out.println("Static block initialization finished!");
}
public static void initialize( ) {
System.out.println("initialize() started!");
ExecutorService executorService = Executors.newSingleThreadExecutor(); // will replace with fixedThreadPool
Future<String> future = executorService.submit(() -> "Hello world!");
System.out.println("Retrieve the result of the future");
String result = null;
try {
result = future.get();
System.out.println(result);
} catch( InterruptedException e ) {
e.printStackTrace();
} catch( ExecutionException e ) {
e.printStackTrace();
}
executorService.shutdown();
}
public static boolean isValid( ) {
return true;
}
}
However, in the above case blocking operation future.get is getting blocked forever, even though it does nothing but a trivial task to return a String.
I also tried using ForkJoinPool, I had no luck.
I monitored threads using jconsole could not detect any deadlock. why is it behaving weirdly?
The static initializer for your Cache class doesn't complete – it's waiting on the completion of future.get(). You could remove the static initializer and call your method directly – Cache.initialize() – from main() or somewhere else, but whatever thread does that is going to be similarly blocked.
I suggest you create a separate thread to call initialize(), thus avoiding the blocking behavior, like this:
new Runnable() {
#Override
public void run() {
initialize();
}
}.run();
This seems to be the expected behavior. this is a classic class initialization deadlock.
a new Thread is started using the Runnable that is dependent on the completion of static initialization of a class. The class is, in turn, waiting for the Runnable to finish because of the future.get() method call.
The static initialization is waiting for thread to complete and thread is waiting for static initialization to complete.
JLS:: Class initialiization gives details on the class initialization process.
I wonder why jconsole couldn't detect the deadlock
Related
I have an application with 2 threads (the main and another thread t1) which share a volatile variable myVar. Any ideas on how to make the main thread to call a method myMethod by signaling in some way from t1 ?
I implemented it by using ChangeListener and myMethod is called when myVar changes, BUT the method is called from t1, and not from the main thread (note: I need to call this method from the main thread because this is a call to a JavaScript code from Java, so for a security reason only the main thread can do so). Thanks in advance.
You would have to have your main thread spin in a loop on some scalar, I would recommend one of the Atomics that java provides (http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/atomic/package-summary.html), but you could use volatile if you wanted for this I think.
Each thread can only run sequentially - it's just the way computing works. The way you will handle this, is when the main thread spins in some sort of loop, you eventually check to see if this scalar of yours has been set, and when it has, you want unset the variable and execute your JavaScript. In this particular piece of your code, I think the Atomics have an advantage over the volatile with the use of the compareAndSet operations because using volatile can mess you up a bit between threads if you are trying to check the value in one operation and then set it again in another operation which gives the other thread enough time to set it again - meaning you may miss a call to your JS because the other thread set the variable between the main thread checking it and setting it (although the use of volatile vs Atomics may be interpreted as my opinion).
//main thread
AtomicBoolean foo = new AtomicBoolean(false);
while (...somecondition...){
if(foo.compareAndSet(true, false)){
//execute JS
}
//do some other work
}
and in your T1 thread, just call foo.set(true).
If you expect main to call your JS for each time T1 sets foo to true, then you will have to block in T1 until main has unset foo, or use an AtomicInteger to count how many times T1 has set foo - depending on your needs.
Since both tread sharing the same instance of myVar, you can make both thread to synchronize on the shared variable. Have main to wait on myVar notification before executing myMethod. Later, t1 can notify through variable myVar, and the waiting thread can continue and proceed with the method call.
The following snippet fully demonstrated the idea
public class MainPlay {
public static void main(String[] args) {
MainPlay mp = new MainPlay();
mp.execute();
}
public void execute() {
Thread main = new Thread(mainRunnable, "main");
Thread t1 = new Thread(t1Runnable, "t1");
main.start();
t1.start();
}
public Object myVar = new Object();
public void myMethod() {
System.out.println("MyMethodInfoked.");
}
public Runnable t1Runnable = new Runnable() {
public void run() {
synchronized(myVar) {
try {
System.out.println("[t1] sleep for 1 sec");
Thread.sleep(1000);
System.out.println("[t1] Notifying myVar so Main can invoke myMethod");
myVar.notify();
} catch (InterruptedException e) {
// interupted.
}
}
}
};
public Runnable mainRunnable = new Runnable() {
public void run() {
synchronized(myVar) {
try {
System.out.println("[main] Waiting for t1 to notify...");
myVar.wait();
} catch (InterruptedException e) {
// interrupted.
}
System.out.println("[main] executing main method");
myMethod();
}
}
};
}
And the output is
[main] Waiting for t1 to notify...
[t1] sleep for 1 sec
[t1] Notifying sharedObject so Main can invoke myMethod
[main] executing main method
MyMethodInfoked.
You could use wait/notify blocks to prevent the main thread from continuing until signalled to do so.
static Main main = // ...
static boolean signal = false;
// t1:
// Do work
signal = true;
synchronized (main) {
main.notify();
}
// main:
synchronized (main) {
while (!signal) {
main.wait();
}
}
myMethod();
In case the main thread has nothing else to do, the approach proposed by #searchengine27 results in unnecessary processor load generated by this thread.
So instead going with some AtomicXXX class it would be better to use some of the blocking queues which allow writing of data from one thread (with put()) and consumption of that data by the other. The main queue would block (by calling take() method) if such a queue is empty not using any CPU resources.
I have a thread pool on the function that the clients calling.. to make only (n) clients execute this function upload() and the others wait.. i tried to call sleep() in the implementation of the function but it didn't work ...
note: I'm doing this to have time to see that other clients doesn't execute the function while there are (n) clients execute it...
i need fast help please ..
the code of Server:
public class Server extends UnicastRemoteObject implements ExcutorInterface
{
public Server()throws RemoteException
{
System.out.println("Server is in listening mode");
}
public static void main(String arg[]) throws InterruptedException
{
try{
LocateRegistry.createRegistry(1234);
Server p=new Server();
Naming.bind("//127.0.0.1:1234/obj",p);
}catch(Exception e)
{
System.out.println("Exception occurred : "+e.getMessage());
}
}
#Override
public void executeJob() throws RemoteException {
System.out.println("Inside executeJob...");
doJob a=new doJob("req_id","usrname","pwd");
ExecutorService threadExecutor = Executors.newFixedThreadPool(2);
threadExecutor.execute(a);
threadExecutor.shutdown();
}
}
the code of doJob :
public class doJob implements Runnable {
String request_id="", usrnamee="", pswd="";
public static int i = 1;
public doJob(String request_id,String usrnamee,String pswd) {
this.request_id=request_id;
this.usrnamee=usrnamee;
this.pswd=pswd;
}
public void upload() throws InterruptedException, IOException {
Thread.sleep(1000*15);
}
public void run() {
upload();
}
}
and I call executeJob(); in the client
One suggestion is to make "threadExecutor" a static member variable of
server.
If you want only n clients then make the pool have n threads
ExecutorService threadExecutor = Executors.newFixedThreadPool(n);
Shutting down within execute method id does not seem right.
The pool should be shutdown only when you decide to shutdown the
server.
Till then it should be alive and process the client requests.
So you have to remove the shutdown and newFixedThreadPool statements
out of the executeJob method.
To elaborate on my comment, you should surround the Thread.sleep in a try/catch and make sure the thread sleeps as long as you wish it to do so. It would look something like this:
long wakeTime = new Date().getTime() + (1000 * 15);
while ((new Date()).getTime() < wakeTime) {
try {
Thread.sleep(1000*15);
} catch (InterruptedException e) {
// do nothing
}
}
I suspect your thread was waking early because of a signal perhaps because of your call to threadExecutor.shutdown() immediately after threadExecutor.execute(a). You might want to consider calling threadExecutor.awaitTermination() as well.
Edits after learning that the task never executes:
Because threadExecutor.shutdown() doesn't wait for the tasks to complete, it looks like your program is immediately exiting. You should try using threadExecutor.awaitTermination() after your call to threadExecutor.shutdown(), placing it in a loop similar to the one suggested for Thread.sleep().
Get rid of the thread pool and use a counting semaphore to control inline execution of the upload.
I hope Thread.sleep() will help you to resolve.
Also you can use wait().
I have an application with a main class that sets up a thread executor for a few other runnable classes however I want an update method in the main class to be called regularly also so is it best to create a thread like in the example below OR submit the class to the thread executor declared inside it (something like in the example below the example)?
Feels wrong using a mixture of thread executors and starting standard threads.
Use standard thread call for main classes updates?
public class Test {
private ScheduledExecutorService scheduledThreadPool; //used for creating other threads
private Thread t;
public Test() {
t = new Thread() {
#Override
public void run() {
try {
while (true) {
processUpdates();
Thread.sleep(10);
}
} catch (InterruptedException e) {
logger.error(e);
}
}
};
}
private void processUpdates() {
//do some stuff
}
}
OR use thread executor for not just the other runnable classes but the main class itself?
public class Test implements runnable {
ScheduledExecutorService scheduledThreadPool = Executors.newScheduledThreadPool(3);
public Test() {
scheduledThreadPool.scheduleWithFixedDelay(this, 0, 10, TimeUnit.MILLISECONDS);
}
# Override
public void run() {
processUpdates();
}
private void processUpdates() {
//do some stuff
}
}
Thanks!
Always use thread pools over plain old threads: it gives you much more control over the execution of your threads.
If you want all your threads to run in parallel, you can always use an unlimited thread pool, which is discouraged because Thread is costly in memory.
In your case, the use of a ScheduledExecutorService is even more recommended since it avoids the sleep instruction in your thread implementation. It gives better performance and a much better readability.
Trying to wrap my head around this code. When I run this - the output will be Roger. Isn't msg a static variable and at a class level thus should print Moore?
EDIT : I've allowed a sleep too allow the child thread to run its course. It also prints printing... Still No Change
public class Test2 {
private static String msg = "Roger";
static {
new Thread(new Runnable() {
public void run() {
System.out.println("printing..");
msg += "Moore";
}
}).start();
}
static {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
}
public static void main(String argv[]) {
System.out.println(msg);
}
}
Trying to wrap my head around this code. When I run this - the output will be Roger. Isn't msg a static variable and at a class level thus should print Moore?
As others have pointed out, this is a race condition but it's more complicated then this simple answer.
EDIT : I've allowed a sleep too allow the child thread to run its course. It also prints printing... Still No Change
When a class is initialized, the static code is executed in the thread that accesses the class first – in this case the main thread. All other threads have to wait for this initialization to complete before they can access the class. This means that the background thread actually stops and waits for the class initialization to complete before it can execute msg += "Moore";. Then it is a race to see whether the msg is assigned to "Roger" and the background thread can append to it before main prints it. Even with the msg field being volatile, the race still exists. You can get a glimpse into the complexities of the process from the JLS section 12.4.2 on Detailed Initialization Procedure.
So what is happening is approximately:
The main thread initializes the Test2 class.
The msg is initialized first because it comes before the static blocks.
First static block is executed which forks the background thread.
Second static block is executed which does the sleep() blocking the initializing thread.
Background thread starts to run (could be before the previous step). It goes to update msg but the class is locked since the main thread is sleeping and hasn't completed with the class initialization. The background thread has to wait.
The main thread wakes up and finishes the initialization.
This releases the block on the class which allows the background thread to continue.
At the same time as the previous step, main is called and it is a race condition to see if the msg can be updated before it is printed out.
In general, forking background threads in static methods like this is extremely frowned upon. Putting a sleep in a static block is obviously not recommended as well.
The main method will not be called till all the static initializers in your class are done. So it will always wait till the static inits are done. Even if there is a sleep in it.
Additionaly static initialization is thread safe, so your forked thread cannot access the variable, till the static init blocks are done.
It's a race condition. There's no guarantee when the Runnable will have executed.
EDIT: This answers responds to the original posted question, in which no delay was present in the static initializer. This was leading to a simple race condition between the main thread reading the static member and the spawned thread updating it.
Rather than wait a little bit, and hope the other thread runs, you can guarentee it with some synchronization:
public class Test {
private static String msg = "Roger";
private static volatile boolean done = false;
private static final Object lock = new Object();
static {
new Thread(new Runnable() {
public void run() {
synchronized(lock)
{
lock.notify();
System.out.println("printing..");
msg += "Moore";
done=true;
}
}
}).start();
}
public static void main(String argv[]) {
synchronized(lock)
{
while(!done)
{
try {
lock.wait();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
System.out.println(msg);
}
}
If the main thread aquires the lock first, then it will msg.wait. It will not continue until notify is called(actually, it continues when the synchronized block containing notify finishes). If the new thread aquires the lock first, then the main thread will have to wait at the start of it's synchronization block. Once it gets in, done will be true. It will not wait, and go straight through.
I have the following piece of code:
public class Test {
List<Future> future = new ArrayList<Future>();
public static void main(String args[]) throws Exception {
Adapter b1 = new Adapter();
final ExecutorService threadPool = Executors.newCachedThreadPool();
for(//iterate for number of files) {
while(data exists in file) {
//Call a function to process and update values in db
future.add(threadPool.submit(new Xyz(b1)));
//read next set of data in file;
}
}
try {
for(Future f: future) {
f.get();
}
}
catch(Exception e) {
throw e;
}
}
}
class Xyz implements Runnable {
private Adapter a1;
public Xyz(Adapter al) {
this.a1=a1;
}
#Override
public void run() {
try {
a1.abc();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
When the number of files is 1 (for loop runs for 1 time), the code runs fine.
But, when the number of files increases, the code never returns back from future.get() method.
just out of curiosity.. do i need to shutdown the executor somewhere ??
Yes, and this is likely the problem. Each Future.get() will block until the corresponding task is complete, then once all the tasks are complete your main thread will exit. But your java process will not exit because the thread pool threads are still active in the background. You should shut down the executor once you have finished with it, most likely as the last thing in your main method.
I also note that you're submitting many tasks that wrap the same Adapter instance and all call its abc() method - check that there's nothing in there that will deadlock when called simultaneously in more than one thread.
Your Callable::call / Runable::run does not return. Otherwise the corresponding future would not block.
Additional executor.shutdown or future.cancel will thow an InterruptedException to stop the thread processing the object you submitted but it is up to you if to catch it or not. Your are responsible for making the jobs you submitted stop.
When you submit thousands Callables/Runnables to a CachedExecutor that it might spawn so many threads that your machine gets so slow that you think it takes forever. But you would have noticed that.
When dealing with an undefined number of parallelizable tasks i suggest to use a FixedThreadPool with not much more threads that there are cpu cores.
Edit: Therefore when you set a breakpoints at a1.abc(); and step forward you will probably find out that it never returns.