I need to perform 2 tasks independently.
First Task
Once per minute it should check whether there is any file in a specific folder. If there is, it should add the names of the files to a queue.
This can be done as follows:
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
public class schedulerExample extends Thread{
public void checkFile()
{
System.out.println("checking whether file exist in folder");
}
public void getFiles()
{
System.out.println("getting the file names");
}
public void enqueueFiles()
{
System.out.println("add files to queue");
}
public static void main(String[] args) {
final schedulerExample obj = new schedulerExample();
ScheduledExecutorService executor = new ScheduledThreadPoolExecutor(1);
executor.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
obj.checkFile();
obj.getFiles();
obj.enqueueFiles();
}
}, 0, 1, TimeUnit.MINUTES);
}
}
Second Task
If the queue is empty , sleep for one minute or else process the file from the queue one-by-one.
public class processModel extends Thread{
public static void getQueueSize(int size)
{
System.out.println("getting queue size");
}
public void dequeue()
{
// dequeue the queue
System.out.println("dequeue");
}
public void processFile()
{
// process the file
System.out.println("process file");
}
public static void main(String[] args) {
final boolean flag = true;
final int size = 9;
final processModel obj = new processModel();
Thread t1 = new Thread(){
public void run()
{
while(flag)
{
obj.dequeue();
obj.processFile();
getQueueSize(size);
if(size == 0)
{
try
{
Thread.sleep(60000);
}
catch(InterruptedException e)
{
}
}
}
}
};
t1.start();
}
}
Now I need to do both in a single class concurrently. Is that possible?
One thread should be fetching the files once per minute. Another thread should execute files one-by-one.. if no files are there it waits for a minute and checks again. In the second method I have used an infinite loop -- instead of that, is there a way that I can perform things one-by-one?
You might want to consider using Callables with the java ExecutorService. This can be used to easily break up tasks and allow them to run concurrently. Beyond that, you can get a Future which will allow you to check the results at any time (or postpone if it's not done).
There is a great book about java and concurrency called "Concurrency in Practice."
Beyond that, Java 7 has new functionality to allow file listeners on directories. That might allow you to abstract this "check and loop" functionality.
Synchronize on the queue object when you get file from it and when you add file to it.
In the thread that reads, call wait() if the queue is empty.
In the thread that checks for new files, call notify() after you added the new file to the queue.
This is how it's usually done.
You should also prevent adding file that is being processed to the queue.
Related
In my applications there are an n number of actions that must happen, one after the other in sequence, for the whole life of the program. Instead of creating methods which implement those actions and calling them in order in a while(true) loop, I decided to create one thread for each action, and make them execute their run method once, then wait until all the other threads have done the same, wait for its turn, and re-execute again, and so on...
To implement this mechanism I created a class called StatusHolder, which has a single field called threadTurn (which signifies which thread should execute), a method to read this value, and one for updating it. (Note, this class uses the Singleton design pattern)
package Test;
public class StatusHolder
{
private static volatile StatusHolder statusHolderInstance = null;
public static volatile int threadTurn = 0;
public synchronized static int getTurn()
{
return threadTurn;
}
public synchronized static void nextTurn()
{
System.out.print("Thread turn: " + threadTurn + " --> ");
if (threadTurn == 1)
{
threadTurn = 0;
}
else
{
threadTurn++;
}
System.out.println(threadTurn);
//Wake up all Threads waiting on this obj for the right turn to come
synchronized (getStatusHolder())
{
getStatusHolder().notifyAll();
}
}
public static synchronized StatusHolder getStatusHolder()
{//Returns reference to this object
if (statusHolderInstance == null)
{
statusHolderInstance = new StatusHolder();
}
return statusHolderInstance;
}
}
Then I have, let's say, two threads which must be execute in the way explained above, t1 and t2.
T1 class looks like this:
package Test;
public class ThreadOne implements Runnable
{
#Override
public void run()
{
while (true)
{
ThreadUtils.waitForTurn(0);
//Execute job, code's not here for simplicity
System.out.println("T1 executed");
StatusHolder.nextTurn();
}
}
}
And T2 its the same, just change 0 to 1 in waitForTurn(0) and T1 to T2 in the print statement.
And my main is the following:
package Test;
public class Main
{
public static void main(String[] args) throws InterruptedException
{
Thread t1 = new Thread(new ThreadOne());
Thread t2 = new Thread(new ThreadTwo());
t1.start();
t2.start();
}
}
So the run method goes like this:
At the start of the loop the thread looks if it can act by checking the turn value with the waitForTurn() call:
package Test;
public class ThreadUtils
{
public static void waitForTurn(int codeNumber)
{ //Wait until turn value is equal to the given number
synchronized (StatusHolder.getStatusHolder())
{
while (StatusHolder.getTurn() != codeNumber)
{
try
{
StatusHolder.getStatusHolder().wait();
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}
}
}
}
If the two values are equal, the thread executes, otherwise it waits on the StatusHolder object to be awaken from the nextTurn() call, because when the turn value changes all the threads are awaken so that they can check if the new turn value is the one they are waiting for so they can run.
Note thatnextTurn() cycles between 0 and 1: that is because in this scenario I just have two threads, the first executes when the turn flag is 0, and the second when its 1, and then 0 again and so on. I can easily change the number of turns by changing this value.
The problem: If I run it, all goes well and seems to work, but suddenly the output console stops flowing, even if the program doesn't crash at all. I tried to put a t1.join() and then a print in the main but that print never executes, this means that the threads never stop/dies, but instead they remain locked sometimes.
This looks to be even more evident if I put three threads: it stops even sooner than with two threads.
I'm relatively new to threads, so I might be missing something really stupid here...
EDIT: I'd prefer not to delete a thread and create a new one every time: creating and deleting thousands of objs every second seems a big work load for the garbage collector.
The reason why I'm using threads and not functions is because in my real application (this code is just simplified) at a certain turn there actually are multiple threads that must run (in parallel), for example: turn 1 one thread, turn 2 one thread, turn 3 30 threads, repeat. So I thought why not creating threads also for the single functions and make the whole think sequential.
This is a bad approach. Multiple threads allow you to execute tasks concurrently. Executing actions "one after the other in sequence" is a job for a single thread.
Just do something like this:
List<Runnable> tasks = new ArrayList<>();
tasks.add(new ThreadOne()); /* Pick better names for tasks */
tasks.add(new ThreadTwo());
...
ExecutorService worker = Executors.newSingleThreadExecutor();
worker.submit(() -> {
while (!Thread.interrupted())
tasks.forEach(Runnable::run);
});
worker.shutdown();
Call worker.shutdownNow() when your application is cleanly exiting to stop these tasks at the end of their cycle.
you can use Semaphore class it's more simple
class t1 :
public class t1 implements Runnable{
private Semaphore s2;
private Semaphore s1;
public t1(Semaphore s1,Semaphore s2){
this.s1=s1;
this.s2=s2;
}
public void run()
{
while (true)
{
try {
s1.acquire();
} catch (InterruptedException ex) {
Logger.getLogger(t1.class.getName()).log(Level.SEVERE, null, ex);
}
//Execute job, code's not here for simplicity
System.out.println("T1 executed");
s2.release();
}
}
}
class t2:
public class t2 implements Runnable{
private Semaphore s2;
private Semaphore s1;
public t2(Semaphore s1,Semaphore s2){
this.s1=s1;
this.s2=s2;
}
public void run()
{
while (true)
{
try {
s2.acquire();
} catch (InterruptedException ex) {
Logger.getLogger(t2.class.getName()).log(Level.SEVERE, null, ex);
}
//Execute job, code's not here for simplicity
System.out.println("T2 executed");
s1.release();
}
}
}
class main:
public class Testing {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
Semaphore s2=new Semaphore(0);
Semaphore s1=new Semaphore(1);
Thread th1 = new Thread(new t1(s1,s2));
Thread th2 = new Thread(new t2(s1,s2));
th1.start();
th2.start();
}}
I am getting StackOverflowError exception report while calling this recursive method :
private void downloadFiles(int index) {
if (index < totalFiles) {
downloadSingleFile(index, new DownloadCallback() {
#Override
public void onSuccess(String filePath) {
downloadFiles(index + 1);
}
});
}
}
I want to ask if I use a Runnable like this way:
int index = 0;
handler = new Handler();
Runnable runnable = new Runnable() {
#Override
public void run() {
downloadFiles();
}
};
handler.post(runnable);
private void downloadFiles() {
if (index < totalFiles) {
downloadSingleFile(index, new DownloadCallback() {
#Override
public void onSuccess(String filePath) {
index ++;
handler.post(runnable);
}
});
}
}
Will this be a recursivity as well and throw exception ?
Thanks
Your current use of recursion sort of defeats the purpose of using multiple threads. Currently, you only create a single thread which will enter downloadFiles(), and will then recursively try to download every file available. This is not really multithreading, it's single threading with recursion. There are several drawbacks to this approach. First, you are not taking advantage of the ability for multiple threads to do work in parallel. Second, since each subsequent recursive call is dependent on the previous one having succeeded, you are trying to download files in serial. If a given file download were to fail, it would break the rest of the recursive chain.
A better approach would be to spawn a new thread for each file download. This would allow you to use the power of multithreading to split the task in parallel, and it also allows progress to continue even if one thread were to encounter some problems.
Have a look at the following code snippet for an idea on how to approach your problem:
public class FileDownloader implements Runnable {
private index;
public FileDownloader(int index) {
this.index = index;
}
public void run() {
downloadSingleFile(index, new DownloadCallback() {
#Override
public void onSuccess(String filePath) {
// this may no longer be needed
}
});
}
}
// use a thread pool of size 5 to handle your file downloads
ExecutorService executor = Executors.newFixedThreadPool(5);
for (int index=0; index < totalFiles; ++index) {
Runnable r = new FileDownloader(index);
executor.execute(r);
}
// shut down the thread pool executor and wait for it to terminate
executor.shutdown();
while (!executor.isTerminated()) {
}
I have the following piece of code which I expected to print "DONE" at the end. But when I ran, "DONE" was never printed and the JVM in fact never terminated.
What did I do wrong?
// File: Simple.java
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
public class Simple {
public static void main(String[] args) throws Exception{
doTest(3);
}
private static void doTest(final int times) {
ScheduledExecutorService tp = Executors.newScheduledThreadPool(times);
Thread[] runnables = new Thread[times];
for (int i = 0; i < runnables.length; ++i) {
runnables[i] = new Thread(new MyRunnable());
}
// schedule for them all to run
for (Thread t : runnables) {
tp.schedule(t, 1, TimeUnit.SECONDS);
}
try {
tp.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
System.out.println("DONE!");
}catch (InterruptedException e) {
e.printStackTrace();
}
}
static class MyRunnable implements Runnable {
#Override
public void run() {
System.out.println("hello world");
}
}
}
There are two things that you're doing wrong here.
First off, if you're using an ExecutorService, you shouldn't then also be creating your own threads. Just submit Runnables to the executor directly - the executor service has its own collection of threads, and runs anything you submit on its own threads, so the threads you created won't even get started.
Second, if you're done with an ExecutorService, and are going to wait until it's terminated, you need to call shutdown() on the executor service after you submit your last job.
Executors.newScheduledThreadPool(times) uses Executors.defaultThreadFactory() for its ThreadFactory.
Here is the documentation
When a Java Virtual Machine starts up, there is usually a single
non-daemon thread (which typically calls the method named main of some
designated class). The Java Virtual Machine continues to execute
threads until either of the following occurs:
The exit method of class Runtime has been called and the security manager has permitted the exit operation to take place.
All threads that are not daemon threads have died, either by returning from the call to the run method or by throwing an exception
that propagates beyond the run method.
So, here is what you had; but I changed 3 things.
Added the shutdown hook
Made the schedule delayed by an extra second for each to demo that the shutdown is being called even before some of the threads are being called to run by the scheduler.
Lastly, you were telling it to wait forever using the Long.MAX. But I think you were doing it because the shutdown feels like it would shutdown now. But it won't.
public static void main(String[] args) throws Exception {
doTest(3);
}
private static void doTest(final int times) {
ScheduledExecutorService tp = Executors.newScheduledThreadPool(times);
Thread[] runnables = new Thread[times];
for (int i = 0; i < runnables.length; ++i) {
runnables[i] = new Thread(new MyRunnable());
}
// schedule for them all to run
int i = 1;
for (Thread t : runnables) {
tp.schedule(t, i++, TimeUnit.SECONDS);
}
System.out.println("Calling shutdown");
tp.shutdown();
}
static class MyRunnable implements Runnable {
#Override
public void run() {
System.out.println("hello world");
}
}
Hope that answers your question. Now, you're kinda duplicating stuff.
If you are going to use the executerservice, you should just tell it to schedule stuff for you.
public static void main(String[] args) throws Exception {
doTest(3);
}
private static void doTest(final int times) {
ScheduledExecutorService tp = Executors.newScheduledThreadPool(times);
for (int i = 0; i < times; ++i) {
tp.schedule(new MyRunnable(), 1, TimeUnit.SECONDS);
}
System.out.println("Calling shutdown");
tp.shutdown();
}
static class MyRunnable implements Runnable {
#Override
public void run() {
System.out.println("hello world");
}
}
You forgot to shutdown your ExecutorService:
tp.shutdown(); // <-- add this
tp.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
Also I should mention that creating Threads to use them as Runnables is not only meaningless, but can be misleading as well. You should really use Runnable instead of Thread for runnables variable.
Runnable[] runnables = new Runnable[times];
I have a requirement in which I have to check whether any file exists in the folder. If yes, then I need to process it one by one. With my basic knowledge I have arrived to a code structure which I have posted below.
I'm creating an infinite loop and checking whether the file exists in that folder. If yes, then I'm creating a thread and processing it, else it waits for a min and checks again.
class sample {
synchronized int getNoOfFiles() {
// get number of files in the folder
}
synchronized void openFile() {
// open one file
}
synchronized void getFileContents() {
// get the file content
}
synchronized void processFileContent() {
//performing some operation on file contents
}
synchronized void closeFile() {
//closing the file
}
synchronized void deleteFile() {
//delete the file
}
}
class Test {
public static void main(String args[]) {
int flag=0;
Sample obj = new Sample();
while(1) {
flag = obj.getNoOfFiles();
if(flag) {
for(i=0;i<flag;i++) {
MyThread1 t1 = new MyThread1() {
public void run() {
obj.openFile();
obj.getFileContents();
obj.processFileContent();
obj.closeFile();
obj.deleteFile();
}
};
t1.start();
}
}
else {
try {
Thread.sleep(60000);
}
}
}
}
}
Instead of doing this kind of thing yourself, I suggest you take a look at the Timer class, that can be used to do recurring tasks. Because messing around with threads manually can often result in weird bugs.
Even better would be ScheduledThreadPoolExecutor, but it might be a bit complicated if you've never used executors before. See Keppil's answer for how to do this.
The differences between the two are nicely summed up here: Java Timer vs ExecutorService?.
I would suggest using a ScheduledExecutorService:
ScheduledExecutorService executor = new ScheduledThreadPoolExecutor(1);
executor.scheduleAtFixedRate(new Runnable() {
public void run()
{
obj.openFile();
obj.getFileContents();
obj.processFileContent();
obj.closeFile();
obj.deleteFile();
}
}, 0, 1, TimeUnit.MINUTES);
Can anyone give me the working example of reading two files simultaneously through threads? and also what would be the best way to read them at one time.
public static void main(String args[]) {
new Runnable(new File("D:/test1.log"),"thread1");
new Runnable(new File("D:/test2.log"),"thread2");
}
private static class Runnable implements Runnable {
private File logFilePath;
Thread runner;
// you should inject the file path of the log file to watch
public Runnable(File logFilePath,String threadName) {
this.logFilePath = logFilePath;
runner = new Thread(this, threadName);
runner.start();
}
_____READ LOGIC HERE____
}
Program that generates logs.I am not using any close or flush .
public final class Slf4jSample {
static Logger logger = LoggerFactory.getLogger(Slf4jSample.class);
static int i=0;
public static void main(final String[] args) {
int delay = 0; // delay for 5 sec.
int period = 10000; // repeat every sec.
Timer timer = new Timer();
timer.scheduleAtFixedRate(new TimerTask() {
public void run() {
// Task here ...
logger.error("error"+i);
logger.warn("warn"+i);
logger.debug("debug"+i);
try{int i=0/0;
}catch(Exception e){
logger.error("Ecxeption"+i, e);
}
i++;
}
}, delay, period);
}
}
I'm not quite sure what your aim is from the short description, but I just want to warn you that reading files in parallel from a single hard disk is generally not a good idea, because the mechanical disk head needs to seek the next reading location and so reading with multiple threads will cause it to keep bouncing around and slow things down instead of speeding them up.
In case you want to read portions of files and process them in paralle, I recommend you to look at a single producer (to read the files sequentially) multiple consumer (to process the files) solution.
Edit: If you insist on using multiple readers, you should probably change your code like this:
public static void main(String args[]) {
new Thread(new ThreadTask(new File("D:/test1.log")),"thread1").start();
new Thread(new ThreadTask(new File("D:/test2.log")),"thread2").start();
}
private static class ThreadTask implements Runnable {
private File logFilePath;
// you should inject the file path of the log file to watch
public ThreadTask(File logFilePath) {
this.logFilePath = logFilePath;
}
public void run() {
// read file
}
}
You have to instantiate a thread object like Thread t = new Thread(new Runnable(.....))
and pass runnable object in Thread's constructor.
Then calling start method on thread object will start separate thread and calls Runnable's run method.
You shouldn't be creating new threads inside Runnable's constructor.