Here's my problem:
I have a whole bunch of identical objects. These objects interface with a server. It takes 4 or 5 seconds to get data from the server, as the server is slow.
Now i need all the objects to get data. So i call MyObject.getData() for each object. I could do it in a series but 20 objects, each taking 5 seconds is too slow. I thought I should use threads and have each object on its own thread.
Here's my question:
If i make the objects extend thread. Will a call to o MyObject.getData(); run in that object's thread, or in the thread the method was called from? I know i can use Thread.Run() to get the object going but thats not what i want. I want to get methods running at my will.
So how do i do this?
Thanks so much.
The text book way to do this could be something like this:
class GetDataObj implements Callable<Data> {
public Data call(){
//get data
return data;
}
}
then
ExecutorService exec = Executors.newCachedThreadPool();
Set<Callable<Data>> objects = //get objects;
List<Future<Data>> futures = exec.invokeAll(objects);
for(Future<Data>> future : futures){
Data data = future.get();
//do stuff with data
}
exec.shutdown();
Note that when you iterate through futures, the get() method will block until the result is available for that DataObj. If you want to wait until all data are available, this is fine.
If you call object.myMethod(), the method will run in the caller thread.
You have to start() the thread to make it run, not call the run() method.
A think you can do, is rewriting your object so that the myMethod() method launch a new thread. So you can use your objects exactly the same as actually. But if myMethod return something, that change, because you have to wait for the thread to terminate befor
I think it would be best to use a thread pool to get data from the objects. Therefore you would need each object to implement Runnable. See thread pools.
If you pass a reference to a queue to the objects once they have got the data they can place it in the queue. The main thread can then just take the data off the queue when it is ready.
See the producer-consumer pattern.
An example of this is:
BlockingQueue<Data> queue = new BlockingQueue<Data>();
ExecutorService pool = Executors.newFixedThreadPool(5);
//implements Runnable, getting data from this
//places Data object in queue instead of returning it
DataObject obj = new DataObject(queue);
pool.execute(obj); //invokes the run method of the DataObject
Data data = queue.take();
You would need to have a for-loop for more than one object.
Hope this helps.
Related
I have a below class in which add method will be called by multiple threads to populate channelMessageHolder CHM in a thread safe way.
In the same class, I have a backgrond thread which runs every 30 seconds and it calls send method by passing data from the channelMessageHolder.
public class Processor {
private final ScheduledExecutorService executorService = Executors
.newSingleThreadScheduledExecutor();
private final AtomicReference<ConcurrentHashMap<Channel, ConcurrentLinkedQueue<Message>>> channelMessageHolder =
new AtomicReference<>(new ConcurrentHashMap<Channel, ConcurrentLinkedQueue<Message>>());
private Processor() {
executorService.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
send();
}
}, 0, 30, TimeUnit.SECONDS);
}
// this will be called by only single background thread
private void send(ConcurrentHashMap<Channel, ConcurrentLinkedQueue<Message>> messageByChannels) {
for(Entry<Channel, ConcurrentLinkedQueue<Message>> entry : messageByChannels.entrySet()) {
Channel channel = entry.getKey();
ConcurrentLinkedQueue<Message> messageHolder = entry.getValue();
while (!messageHolder.isEmpty()) {
Message message = messageHolder.poll();
....
// process this and send to database
}
}
}
// called by multiple threads
public void add(final Channel channel, final Message message) {
// populate channelMessageHolder in a thread safe way
}
}
Question
As you can see channelMessageHolder is already present in my Processor class so do I need to explicitly pass data from this map every 30 seconds to send method? Or I can directly use it in my send method?
Confusion is, if I directly use it in my send method, then it will be populated by multiple threads at the same time so that's why I am using getAndSet method of AtomicReference to pass it to send method.
Let me know if what I am doing is wrong and there is any better way to do that?
As you can see channelMessageHolder is already present in my Processor class so do I need to explicitly pass data from this map every 30 seconds to send method? Or I can directly use it in my send method?
You can certainly use it directly in the send() method and you don't need the AtomicReference wrapper since ConcurrentHashMap is already synchronized. What you need to worry about is that your key and value objects in the map are being properly synchronized. I assume the Channel is immutable and the ConcurrentLinkedQueue is concurrent so you should be good.
// no need for AtomicReference
private final ConcurrentHashMap<Channel, ConcurrentLinkedQueue<Message>> channelMessageHolder =
new ConcurrentHashMap<Channel, ConcurrentLinkedQueue<Message>>();
The ConcurrentHashMap takes care of the synchronization for you so the producer threads can add items into it at the same time that your sender thread is sending items out without clashing. An AtomicReference is only needed if you were trying to share an unsynchronized class between multiple threads.
Confusion is, if I directly use it in my send method, then it will be populated by multiple threads at the same time so that's why I am using getAndSet method of AtomicReference to pass it to send method.
Right but this ok. Multiple threads will be adding messagesto the ConcurrentLinkedQueue. Every 30 seconds your background thread starts up, gets the Channel, dequeues and then sends the messages that are in the queue at that moment. The ConcurrentLinkedQueue protects against race conditions of the producer and consumer.
The problem that you have in your code is that this isn't reentrant since it relies on multiple calls to the queue:
while (!messageHolder.isEmpty()) {
Message message = messageHolder.poll();
It works in your case because there looks to be only one thread dequeue-ing but the following code is better:
while (true) {
// only one call to the concurrent queue
Message message = messageHolder.poll();
if (message == null) {
break;
}
...
}
Or I can directly use it in my send method without passing anything
You should be able to directly use it in the send method by saying channelMessageHolder.getAndSet(new ConcurrentHashMap<Channel, ConcurrentLinkedQueue<Message>>()) at the beginning of the send method without any issues.
That said, Java 8 has added a new method called computeIfAbsent to the ConcurrentHashMap class which means that you don't really need that AtomicReference that you are using.
I have a code where I create a thread per object at the start. Then I store the threads and the relevant objects in a hashmap. When a particular method is called, I need to get the thread from the hashmap using the object which called that method and execute some logic. Below is my method.
public void sendMessage(Message message) {
for (Iterator<MessageListenerItem> iterator = messageListnerItemList.iterator(); iterator.hasNext();) {
MessageListenerItem messageListnerItem = (MessageListenerItem) iterator.next();
Thread messageListenerThread = threadMessageListenerMap.get(messageListnerItem.getMl());
System.out.println("The thread that is executing is : " + messageListenerThread.getName());
this.messageListenerItem = messageListnerItem;
this.message = message;
messageListenerThread.start();
}
}
So this if a method is received from object "A", then thread "A" will be started. But before the end of execution of thread "A", if object "A" calls this method again, then I will have to wait till the thread "A" end and then use it again for the 2nd call. But in the mean time if Object "B" calls this method it will get its Thread "B" from the hashmap and use it. So if I use wait-notify, how can I notify only Thread B once its execution is complete and not Thread A. How can I achieve this? Please advice.
Wait/notify is too crude a mechanism for what you want to achieve.
You will have to set some flag or something so that ThreadB needs to know that the notify was not addressed to it. And use notifyAll.
Can you use a different monitor for each thread?
I have a class which basically does the same series of steps twice. Sounds like a perfect example of where to multithread your program. My question is though if I can do this with only two threads. Here is the general jist of things
Tester implements Runnable{
Thread obj1Thread, obj2Thread;
MyObj obj1, obj2;
String obj1Results, obj2Results;
void runTests(){
obj1Thread = new Thread(this, "ob1 thread");
obj2Thread = new Thread(this, "ob2 thread");
obj1.start();//builds up obj1
obj2.start();//builds up obj2
if(obj1 and obj2 are finished building){
System.out.println(obj1);
System.out.println(obj2);
}
obj1Thread.startSecondPhase()//runs a separate function that tests obj1 vs ob2. Essentially test(ob1, ob2)
obj2Thread.startSecondPhase()//runs a separate function that tests obj2 vs ob1. Essentially test(ob2, ob1)
if(obj1 and obj2 are finished testing){
System.out.println(obj1Results);
System.out.println(obj2Results);
}
}
}
I have gotten the first part - building up the objects - working. My questions are now -
How can I get the main thread to wait for the two threads to finish their first part? Perhaps the main would do a wait on both objects and then after the threads notifyAll they do a wait on the main thread? But then how do the threads get a hold of the main thread? Perhaps with this?
How can I have this 'second phase' of the run function without making a new class with a new thread and a new specific run function? I dont want to have to make a new class and everything for every little task.
To clarify the sequence of events I want specifically is -
Main thread initializes and starts two threads
Both threads simultaneously build their respective objects
When both threads finish building they pause. Then main thread prints the objects out in order.
After main thread is done, the two threads continue their code to a testing phase simultaneously
When the threads are done the main thread prints the results out. Could probably use a join() here
Edit: Also, how can I tell the specific threads which objects I want them to work on? Right now Im doing it in a kinda hacky way (i'm working off the thread name).
I would use higher-level abstractions: use an execute and ExecutorService.invokeAll(Collection<? extends Callable<T>> tasks), which returns a list of Future.
Your main thread can
dispatch two tasks,
obtain two futures,
print the results, then
dispatch two more tasks
The executor service and futures will handle all the concurrency under the hood.
EDIT:
I see your comment:
A special Runnable class just to implement basically one line of code?
Perhaps I'm being too idealistic but that feels wrong to me.
You typically use ananymous inner classes in such case:
Future future = executorService.submit(new Runnable() {
public void run() {
System.out.println("Asynchronous task");
}
});
Nothing wrong with that. When Java has lambda it will become even shorter.
Future future = executorService.submit(() -> {System.out.println("Asynchronous task");});
I'm doing a download application for android. The downloading part is now successfully implemented and its working.
What I need is to download the file parallel in segments. To be more clear, if the user specify 8 segments, I want to create 8 Threads and do the downloading.
So in what way will I be able to create 8 threads dynamically? Also as I'm doing this for an phone how will I be able to maintain the memory consumption at a minimum level?
I have not worked with threads before, so I hope you can help me with this. Thank you for your time! :)
The most efficient way to create a fixed number of threads is to use the ExecutorService:
ExecutorService exec = Executors.newFixedThreadPool(8);
It's basically a fixed-size thread pool that takes a lot of the management burden from the developer.
Edit: So your flow should be something like this:
First, define your thread task class (each thread will execute the call method of its own task):
class ThreadTask implements Callable<Object> {
public Object call() {
// execute download
...
return result;
}
}
If you want to pass any parameters to the tasks, put some private fields in the class above and pass them through a constructor. Also, you can return any type from call, just change the type in the implements Callable<...> part.
When you want to fire off the threads, create the pool and submit the tasks:
ExecutorService exec = Executors.newFixedThreadPool(8);
List<Future<Object>> results = new ArrayList<Future<Object>>();
// submit tasks
for(int i = 0; i < 8; i++) {
results.add(exec.submit(new ThreadTask()));
}
...
// stop the pool from accepting new tasks
exec.shutdown();
// wait for results
for(Future<Object> result: results) {
Object obj = result.get();
}
Take a look at ExecutorService, in particular Executors.newFixedThreadPool(int i), this is an excellent way to handle threads in a system friendly matter.
I understand the concept behind threading and have written threads in other languages, but I am having trouble understanding how to adapt them to my needs in java.
Basicly at present I have a vector of objects, which are read in from a file sequentially.
The file then has a list of events, which need to happen concurrently so waiting for one event to finish which takes 20-30 seconds is not an option.
There is only a couple of methods in the object which deal with these events. However from looking at tutorials, objects must extend/implement threads/runnable however if the object is in a thread making a method call to that object seems to happen sequentially anyway.
An y extra information would be appreciated as I am clearly missing something I am just not quite sure what!
So to summarise how can I execute a single method using a thread?
To start a thread you call start() on an instance of Thread or a subclass thereof. The start() method returns immediately. At the same time, the other thread (the one incarnated by the Thread instance) takes off, and proceeds with executing the run() method of the Thread instance.
Managing threads is not as easy as it seems. For a smoother API, try using an Executor (see the classes in java.util.concurrent).
The best thing to do in Java is create another class that takes in the data you need to process and performs whatever you need it to perform:
class Worker implements Runnable{
Object mydata;
Worker(Object data)
{
mydata = data;
}
#override
void run()
{
//process the data
System.out.println(data.toString());
//or if you want to use your class then:
YourClass yc = (YourClass)myData;
yc.methodB();
}
}
class YourClass
{
private final ExecutorService executor = Executors.newCachedThreadPool();
private ArrayList<Object> list;
YourClass()
{
list = new ArrayList<Object>();
list.add(new Object());
...
...
list.add(new Object());
}
void methodA()
{
for(Object item : list )
{
// Create a new thread with the worker class taking the data
executor.execute(new Worker(item));
}
}
void methodB(){/*do something else here*/}
}
Note that instead of getting the data, you can pass the actual class that you need the method to be invoked on:
executor.execute(new Worker(new MyClass()));
In the run method of the Worker class you invoke whatever you need to invoke on MyClass... the executor creates a new thread and calls run on your Worker. Each Worker will run in a separate thread and it will be parallel.
Thomas has already given the technical details. I am going to try and focus on the logic.
Here is what I can suggest from my understanding of your problem.
Lets say you have a collection of objects of type X (or maybe even a mix of different types). You need to call methods foo and/or bar in these objects based on some event specified. So now, you maybe have a second collection that stores those.
So we have two List objects (one for the X objects and other for the events).
Now, we have a function execute that will take X, and the event, and call foo or bar. This execute method can be wrapped in a thread, and executed simultaneously. Each of these threads can take one object from the list, increment the counter, and execute foo/bar. Once done, check the counter, and take the next one from the list. You can have 5 or more of these threads working on the list.
So, as we see, the objects coming from file do not have to be the Thread objects.
You have to be very careful that the List and counter are synchronized. Much better data structures are possible. I am sticking to a crude one for ease of understanding.
Hope this helps.
The key to threads is to remember that each task that must be running must be in its own thread. Tasks executing in the same thread will execute sequentially. Dividing the concurrent tasks among separate threads will allow you to do your required cocurrent processing.