Suppose I have a class which implements Runnable interface, and I am to make 5 instances of given class in the main program. I would like to store them either in array, or a collection. Since the class implements Runnable it is my understanding that the only way I can store it is in a thread container such as Thread[]. However if I do this I can't use classes overridden toString() method for example, or any other custom method/field.
public class LittleClass implements Runnable{
public void run(){
}
}
public static void main(String[] args){
Thread[] smallClasses = new Thread[5];
// initialize and so...
smallClasses[i].customField//not accessible
System.out.println(smallClasses[i])//gives Thread[Thread-X,X,]
}
You should consider using an ExecutorService. Then you keep an array of your job classes and submit them to the service to be run.
// create a thread pool with as many workers as needed
ExecutorService threadPool = Executors.newCachedThreadPool();
// submit your jobs which should implements Runnable
for (YourRunnable job : jobs) {
threadPool.submit(job);
}
Once you have submitting your jobs, you shut down the service, wait for it to finish, and then you can interrogate your jobs to get information from them.
// shuts the pool down but the submitted jobs still run
threadPool.shutdown();
// wait for all of the jobs to finish
threadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
// now go back and print out your jobs
for (YourRunnable job : jobs) {
System.out.println(jobs.toString());
}
Here's a good tutorial on the subject.
You can create your custom class which implements Runnable and then story an array of those custom classes.
So, for instance, in the code you wrote above, you can always use
LittleClass[] objs = new LittleClass[4];
for(int i = 0; i < objs.length; i++) {
objs[i] = new LittleClass();
}
Related
I have a BlockingQueue of Runnable - I can simply execute all tasks using one of TaskExecutor implementations, and all will be run in parallel.
However some Runnable depends on others, it means they need to wait when Runnable finish, then they can be executed.
Rule is quite simple: every Runnable has a code. Two Runnable with the same code cannot be run simultanously, but if the code differ they should be run in parallel.
In other words all running Runnable need to have different code, all "duplicates" should wait.
The problem is that there's no event/method/whatsoever when thread ends.
I can built such notification into every Runnable, but I don't like this approach, because it will be done just before thread ends, not after it's ended
java.util.concurrent.ThreadPoolExecutor has method afterExecute, but it needs to be implemented - Spring use only default implementation, and this method is ignored.
Even if I do that, it's getting complicated, because I need to track two additional collections: with Runnables already executing (no implementation gives access to this information) and with those postponed because they have duplicated code.
I like the BlockingQueue approach because there's no polling, thread simply activate when something new is in the queue. But maybe there's a better approach to manage such dependencies between Runnables, so I should give up with BlockingQueue and use different strategy?
If the number of different codes is not that large, the approach with a separate single thread executor for each possible code, offered by BarrySW19, is fine.
If the whole number of threads become unacceptable, then, instead of single thread executor, we can use an actor (from Akka or another similar library):
public class WorkerActor extends UntypedActor {
public void onReceive(Object message) {
if (message instanceof Runnable) {
Runnable work = (Runnable) message;
work.run();
} else {
// report an error
}
}
}
As in the original solution, ActorRefs for WorkerActors are collected in a HashMap. When an ActorRef workerActorRef corresponding to the given code is obtained (retrieved or created), the Runnable job is submitted to execution with workerActorRef.tell(job).
If you don't want to have a dependency to the actor library, you can program WorkerActor from scratch:
public class WorkerActor implements Runnable, Executor {
Executor executor=ForkJoinPool.commonPool(); // or can by assigned in constructor
LinkedBlockingQueue<Runnable> queue = new LinkedBlockingQueu<>();
boolean running = false;
public synchronized void execute(Runnable job) {
queue.put(job);
if (!running) {
executor.execute(this); // execute this worker, not job!
running=true;
}
public void run() {
for (;;) {
Runnable work=null;
synchronized (this) {
work = queue.poll();
if (work==null) {
running = false;
return;
}
}
work.run();
}
}
}
When a WorkerActor worker corresponding to the given code is obtained (retrieved or created), the Runnable job is submitted to execution with worker.execute(job).
One alternate strategy which springs to mind is to have a separate single thread executor for each possible code. Then, when you want to submit a new Runnable you simply lookup the correct executor to use for its code and submit the job.
This may, or may not be a good solution depending on how many different codes you have. The main thing to consider would be that the number of concurrent threads running could be as high as the number of different codes you have. If you have many different codes this could be a problem.
Of course, you could use a Semaphore to restrict the number of concurrently running jobs; you would still create one thread per code, but only a limited number could actually execute at the same time. For example, this would serialise jobs by code, allowing up to three different codes to run concurrently:
public class MultiPoolExecutor {
private final Semaphore semaphore = new Semaphore(3);
private final ConcurrentMap<String, ExecutorService> serviceMap
= new ConcurrentHashMap<>();
public void submit(String code, Runnable job) {
ExecutorService executorService = serviceMap.computeIfAbsent(
code, (k) -> Executors.newSingleThreadExecutor());
executorService.submit(() -> {
semaphore.acquireUninterruptibly();
try {
job.run();
} finally {
semaphore.release();
}
});
}
}
Another approach would be to modify the Runnable to release a lock and check for jobs which could be run upon completion (so avoiding polling) - something like this example, which keeps all the jobs in a list until they can be submitted. The boolean latch ensures only one job for each code has been submitted to the thread pool at any one time. Whenever a new job arrives or a running one completes the code checks again for new jobs which can be submitted (the CodedRunnable is simply an extension of Runnable which has a code property).
public class SubmissionService {
private final ExecutorService executorService = Executors.newFixedThreadPool(5);
private final ConcurrentMap<String, AtomicBoolean> locks = new ConcurrentHashMap<>();
private final List<CodedRunnable> jobs = new ArrayList<>();
public void submit(CodedRunnable codedRunnable) {
synchronized (jobs) {
jobs.add(codedRunnable);
}
submitWaitingJobs();
}
private void submitWaitingJobs() {
synchronized (jobs) {
for(Iterator<CodedRunnable> iter = jobs.iterator(); iter.hasNext(); ) {
CodedRunnable nextJob = iter.next();
AtomicBoolean latch = locks.computeIfAbsent(
nextJob.getCode(), (k) -> new AtomicBoolean(false));
if(latch.compareAndSet(false, true)) {
iter.remove();
executorService.submit(() -> {
try {
nextJob.run();
} finally {
latch.set(false);
submitWaitingJobs();
}
});
}
}
}
}
}
The downside of this approach is that the code needs to scan through the entire list of waiting jobs after each task completes. Of course, you could make this more efficient - a completing task would actually only need to check for other jobs with the same code, so the jobs could be stored in a Map<String, List<Runnable>> structure instead to allow for faster processing.
I have multiple tasks/Runnable (i.e. downloading images from internet), which are generated as the user scrolls through a list in a Android App.
I cannot control how many tasks/Runnable are generated at a time, this could in 100. But I want to execute only n(10) tasks in parallel. So, I am planning to build a design, where as soon as a new task/runnable is generated, it will be added to a queue (List<Runnable>) and through Executors.newFixedThreadPool(10), I will execute only first 10 runnable tasks in parallel. Now as soon as the tasks/Runnable are completed, I should be able to remove them from queue (List<Runnable>) and should be able to execute new tasks/Runnable which are in queue, in FIFO.
I have two classes for this design. First is ExecutorManager which is a singleton class and manages the executions of 10 parallels tasks and second is the ImageDownloader class which implements runnable and is responsible to download the image. I am not sure what is the best way to inform the ExecutorManager that the task/download is completed and it can execute new task from the queue. I am following FIFO, so I will always start execution from first 10 tasks in the queue, but how will I get to know, which task is completed and which one to remove from the Queue?
public class ImageDownloader implements Runnable{
DownloadListener mDownloadListener;
public ImageDownloader(DownloadListener mDownloadListener, String URL){
this.mDownloadListener = mDownloadListener;
}
#Override
public void run() {
//Download the Image from Internet
//ToDo
//if Success in download
mDownloadListener.onDownloadComplete();
//if Error in download
mDownloadListener.onDownloadFailure();
//Inform the Executor Manager that the task is complete and it can start new task
incrementCount();
}
private static synchronized void incrementCount(){
ExecutorManager.getInstance().OnTaskCompleted();// is there a better way to do it
}
}
public class ExecutorManager {
private static ExecutorManager Instance;
ExecutorService executor = Executors.newFixedThreadPool(Constants.NumberOfParallelThread);
ArrayList<Runnable> ExecutorQueue = new ArrayList<Runnable>();
int ActiveNumberOfThread = 0;
private ExecutorManager(){
}
public static ExecutorManager getInstance(){
if(Instance==null){
Instance = new ExecutorManager();
}
return Instance;
}
private void executeTask(){
if(ExecutorQueue.size()>0 && ActiveNumberOfThread < Constants.NumberOfParallelThread){
++ActiveNumberOfThread;
executor.execute(ExecutorQueue.get(0));//Execute the First Task in Queue
}
}
public void enQueueTask(Runnable Task){
ExecutorQueue.add(Task);
executeTask();
}
public void removeFromQueue(){
//How to know, which task to remove?
ExecutorQueue.remove(0);
}
public void OnTaskCompleted(){
--ActiveNumberOfThread;
removeFromQueue();
executeTask();
}
}
Well, you're in luck. You don't have to tell the ExecutorManager anything at all. An ExecutorService with a BlockingQueue handles queues for you. All you have to do is submit the Runnable to the ExecutorService. It will hold on to it. If there are any open threads, it will run it instantly. Else, it will wait until one of the other Runnables finishes execution. Once it finishes, it will take the next one.
If you look at the source code for Executors#newFixedThreadPool, it actually just creates a ThreadPoolExecutor with nThreads threads backed by a LinkedBlockingQueue like so:
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
}
I am pretty new to using multithreading, but I want to invoke a method asynchronously (in a separate Thread) rather than invoking it synchronously. The basic idea is that I'm creating a socket server with an object in memory, so for each client I will have to run something like object.getStuff() asynchronously.
The two constructs I found were:
having the class implement Runnable and threading this and
declaring a runnable class within a method.
Additionally this method needs a return value- will it be necessary to use Executor and Callable to achieve this? Could someone point me in the right direction for implementing this?
I have tried implement option 2, but this doesn't appear to be processing concurrently:
public class Test {
private ExecutorService exec = Executors.newFixedThreadPool(10);
public Thing getStuff(){
class Getter implements Callable<Thing>{
public Thing call(){
//do collection stuff
return Thing;
}
}
Callable<Thing> callable = new Getter();
Future<Thing> future = exec.submit(callable);
return future.get();
}
}
I am instantiating a single test object for the server and calling getStuff() for each client connection.
Threading Tutorial
The Java tutorial on concurrency has a good section on this. It's at https://docs.oracle.com/javase/tutorial/essential/concurrency/runthread.html. Essentially, you can either implement Runnable or Callable, or inherit from Thread.
Subclassing Thread
You can write a class, including an anonymous inner class, that extends Thread. Instantiate it, then invoke the start() method.
public class MyThread extends Thread {
public void run() {
System.out.println("This is a thread");
}
public static void main(String[] args) {
MyThread m = new MyThread();
m.start();
}
}
Implementing Runnable
You can write a class that implements Runnable, then wrap an instance in a Thread and invoke start(). Very much like the previous.
public class MyRunnable implements Runnable {
public void run() {
System.out.println("This is a thread");
}
public static void main(String[] args) {
MyRunnable r = new MyRunnable();
(new Thread(r)).start();
}
}
Return Value
Runnable doesn't allow for return values. If you need that, you need to implement Callable instead. Callable looks a lot like Runnable, except you override the call() method instead of the run() method, and you need to give it to an ExecutorService.
public class MyCallable implements Callable<Integer> {
public Integer call() {
System.out.println("A thread using Callable<Integer>");
return 42;
}
public static void main(String[] args) {
MyCallable c = new MyCallable();
Future<Integer> f = Executors.newSingleThreadExecutor().submit(c));
System.out.println("The thread returned: " +
f.get());
}
}
The two constructs I found were 1) having the class implement Runnable and threading 'this' and 2) declaring a runnable class within a method.
Option (2) probably is better. Most programs would be improved if they had more classes, not fewer. Each named entity in a program—each package, class, method, whatever—should have just one responsibility. In your option (1), you are asking the class to do two things.
For your option (2), you don't actually have to declare a whole class. You can either use an anonymous inner class, or if you can go with Java8 all the way, you can use a lambda expression. Google for either one to learn more.
Additionally this method needs a return value.
The classic way, is for the Runnable object to return the value through one of its own fields before the thread terminates. Then the parent thread, can examine the object and get the return value afterward.
Will it be necessary to use Executor and Callable to achieve this?
Necessary? A lot of people think that ExecutorService is a Good Thing.
Sounds like you are creating a server that serves multiple clients. Do these clients continually connect and disconnect? The advantage of using a thread pool (i.e., ThreadPoolExecutor) is that it saves your program from continually creating and destroying threads (e.g., every time a client connects/disconnects). Creating and destroying threads is expensive. If you have a lot of clients connecting and disconnecting, then using a thread pool could make a big difference in the performance of your server.
Creating and managing threads by yourself is generally bad approach.
As you already pointed - use Executors utility class to create executor and submit Callables to it.
public class RunWResult implements Runable{
private volatile ResultType var;
//the thread method
public void run(){
...
//generate a result and save it to var
var = someResult();
//notify waiting threads that a result has been generated
synchronized(this){
notify();
}
}
public ResultType runWithResult(){
//run the thread generating a result
Thread t = new Thread(this);
t.start();
//wait for t to create a result
try{
wait();
}catch(InterruptedException e){}
//return the result
return var;
}
}
I have been working on writing a multi threaded java program. Before I progress I was thinking how best I can write the program.
I read the differences between runnable and thread and what we should use and what we should not. However I have a question. Is it good to follow the runnable implementation to thread even if the threads are not sharing data i.e the same runnable class object?
I would end up creating different runnable objects thus occupying memory.
Also another idea I have is to pool runnable objects and change the value they hold and assign it to a thread. Thereby having only a set of runnable objects and thus utilizing memory better.
Sample Code:
public class MrRunnable implements Runnable {
private String toFireUrl;
MrRunnable(String url){
}
#Override
public void run() {
// do some function here
}
}
public class Main {
public static void main(String[] args) {
// We will create 500 threads
for (int i = 0; i < 500; i++) {
Runnable task = new MrRunnable("some new url");
Thread worker = new Thread(task);
//start the thread
worker.start();
}
}
}
Here I am creating a new instance of runnable objects and my threads don't share data.
So is this way justified? Or is it better to create a pool of runnable objects and I let my threads manipulate their content and use it?
Each thread you start allocates a call stack, sized by default at 1 Megabyte. Each MrRunnable you create allocates... 24 bytes. Keeping things in perspective helps.
I am trying to figure out how to use the types from the java.util.concurrent package to parallelize processing of all the files in a directory.
I am familiar with the multiprocessing package in Python, which is very simple to use, so ideally I am looking for something similar:
public interface FictionalFunctor<T>{
void handle(T arg);
}
public class FictionalThreadPool {
public FictionalThreadPool(int threadCount){
...
}
public <T> FictionalThreadPoolMapResult<T> map(FictionalFunctor<T> functor, List<T> args){
// Executes the given functor on each and every arg from args in parallel. Returns, when
// all the parallel branches return.
// FictionalThreadPoolMapResult allows to abort the whole mapping process, at the least.
}
}
dir = getDirectoryToProcess();
pool = new FictionalThreadPool(10); // 10 threads in the pool
pool.map(new FictionalFunctor<File>(){
#Override
public void handle(File file){
// process the file
}
}, dir.listFiles());
I have a feeling that the types in java.util.concurrent allow me to do something similar, but I have absolutely no idea where to start.
Any ideas?
Thanks.
EDIT 1
Following the advices given in the answers, I have written something like this:
public void processAllFiles() throws IOException {
ExecutorService exec = Executors.newFixedThreadPool(6);
BlockingQueue<Runnable> tasks = new LinkedBlockingQueue<Runnable>(5); // Figured we can keep the contents of 6 files simultaneously.
exec.submit(new MyCoordinator(exec, tasks));
for (File file : dir.listFiles(getMyFilter()) {
try {
tasks.add(new MyTask(file));
} catch (IOException exc) {
System.err.println(String.format("Failed to read %s - %s", file.getName(), exc.getMessage()));
}
}
}
public class MyTask implements Runnable {
private final byte[] m_buffer;
private final String m_name;
public MyTask(File file) throws IOException {
m_name = file.getName();
m_buffer = Files.toByteArray(file);
}
#Override
public void run() {
// Process the file contents
}
}
private class MyCoordinator implements Runnable {
private final ExecutorService m_exec;
private final BlockingQueue<Runnable> m_tasks;
public MyCoordinator(ExecutorService exec, BlockingQueue<Runnable> tasks) {
m_exec = exec;
m_tasks = tasks;
}
#Override
public void run() {
while (true) {
Runnable task = m_tasks.remove();
m_exec.submit(task);
}
}
}
How I thought the code works is:
The files are read one after another.
A file contents are saved in a dedicated MyTask instance.
A blocking queue with the capacity of 5 to hold the tasks. I count on the ability of the server to keep the contents of at most 6 files at one time - 5 in the queue and another fully initialized task waiting to enter the queue.
A special MyCoordinator task fetches the file tasks from the queue and dispatches them to the same pool.
OK, so there is a bug - more than 6 tasks can be created. Some will be submitted, even though all the pool threads are busy. I've planned to solve it later.
The problem is that it does not work at all. The MyCoordinator thread blocks on the first remove - this is fine. But it never unblocks, even though new tasks were placed in the queue. Can anyone tell me what am I doing wrong?
The thread pool you are looking for is the ExecutorService class. You can create a fixed-size thread pool using newFixedThreadPool. This allows you to easily implement a producer-consumer pattern, with the pool encapsulating all the queue and worker functionality for you:
ExecutorService exec = Executors.newFixedThreadPool(10);
You can then submit tasks in the form of objects whose type implements Runnable (or Callable if you want to also get a result):
class ThreadTask implements Runnable {
public void run() {
// task code
}
}
...
exec.submit(new ThreadTask());
// alternatively, using an anonymous type
exec.submit(new Runnable() {
public void run() {
// task code
}
});
A big word of advice on processing multiple files in parallel: if you have a single mechanical disk holding the files it's wise to use a single thread to read them one-by-one and submit each file to a thread pool task as above, for processing. Do not do the actual reading in parallel as it will degrade performance.
A simpler solution than using ExecuterService is to implement your own producer-consumer scheme. Have a thread that create tasks and submits to a LinkedBlockingQueue or ArrayBlockingQueue and have worker threads that check this queue to retrieve the tasks and do them. You may need a special kind of tasks name ExitTask that forces the workers to exit. So at the end of the jobs if you have n workers you need to add n ExitTasks into the queue.
Basically, what #Tudor said, use an ExecutorService, but I wanted to expand on his code and I always feel strange editing other people's posts. Here's a sksleton of what you would submit to the ExecutorService:
public class MyFileTask implements Runnable {
final File fileToProcess;
public MyFileTask(File file) {
fileToProcess = file;
}
public void run() {
// your code goes here, e.g.
handle(fileToProcess);
// if you prefer, implement Callable instead
}
}
See also my blog post here for some more details if you get stuck
Since processing Files often leads to IOExceptions, I'd prefer a Callable (which can throw a checked Exception) to a Runnable, but YMMV.