I have to call more than one webservice in one method each webservice is executed by separate thread in concurrent/parellel. Every web service will return one ArrayList. Note: may chance some webservices will fail or take more time process response in this case i have to skip these failure result. How can I achieve this? I tried this sample code.
public class MultiThreadsEx{
public class Task implements Runnable {
private Object result;
private String id;
int maxRowCount = 0;
public Task(String id) {
this.id = id;
}
public Object getResult() {
return result;
}
public void run() {
try {
System.out.println("Running id=" + id+" at "+Utilities.getCurrentJavaDate("DD/MM/YYYY HH:MM:SS"));
if(id.equalsIgnoreCase("1")){
/**Getting Details from Amazon WS*/
maxRowCount = AmazonUtils.getweather(cityname);
}else if(id.equalsIgnoreCase("2")){
/**Getting Details from Google WS* /
maxRowCount = GoogleUtils.getWeather(cityName);
}
// call web service
//Thread.sleep(1000);
//result = id + " more";
result = maxRowCount;
} catch (InterruptedException e) {
// TODO do something with the error
throw new RuntimeException("caught InterruptedException", e);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public static void runInParallel(Runnable runnable1, Runnable runnable2) {
try {
Thread t1 = new Thread(runnable1);
Thread t2 = new Thread(runnable2);
t1.start();
t2.start();
} catch (Exception e) {
// TODO do something nice with exception
throw new RuntimeException("caught InterruptedException", e);
}
}
public void foo() {
try {
Task task1 = new Task("1");
Task task2 = new Task("2");
runInParallel(task1, task2);
System.out.println("task1 = " + task1.getResult()+" at "+Utilities.getCurrentJavaDate("DD/MM/YYYY HH:MM:SS"));
System.out.println("task2 = " + task2.getResult()+" at "+Utilities.getCurrentJavaDate("DD/MM/YYYY HH:MM:SS"));
} catch (Exception e) {
//TODO Auto-generated catch block
e.printStackTrace();
}
}
But run() return type is void so how can return result? Examples are highly appreciated. I am new to multithread/concurrent threads concept so if I'm doing anything wrong, please point me in the right direction.
Consider replacing Runnable - run with Callable - call. This will allow you to return a result from your thread task:
public class Task implements Callable<Object> {
private Object result;
public Object call() {
// compute result
return result;
}
}
Now use an ExecutorService:
public static void runInParallel(Callable<Object> c1, Callable<Object> c2) {
ExecutorService exec = Executors.newFixedThreadPool(2);
Future<Object> f1 = exec.submit(c1);
Future<Object> f2 = exec.submit(c2);
}
Later in the code you can use f1.get() and f2.get() to wait for the results of the tasks.
The usual way to communicate the results of a Runnable back to the object which created it is by passing the creating object to the constructor of the Runnable. When the task is finished, you can call a method in the creating object and pass the result data.
Related
I had an existing non-parallel code that we recently made concurrent by using executor service. Adding concurrency ensured limit the number of requests sent to another API in our scenario. So we are calling an external service and limiting requests, waiting for all requests to complete so as merge the responses later before sending the final response.
I am stuck on how to add a unit test/mock test to such a code, considering the private method is parallelized. I have added below my code structure to explain my situation.
I am trying to test here
#Test
public void processRequest() {
...
}
Code
int MAX_BULK_SUBREQUEST_SIZE = 10;
public void processRequest() {
...
// call to private method
ResponseList responseList = sendRequest(requestList);
}
private void sendRequest(List<..> requestList) {
List<Response> responseList = new ArrayList<>();
ExecutorService executorService = Executors.newFixedThreadPool(10);
int numOfSubRequests = requestList.size();
for (int i = 0; i < numOfSubRequests; i += MAX_BULK_SUBREQUEST_SIZE) {
List<Request> requestChunk;
if (i + MAX_BULK_SUBREQUEST_SIZE >= numOfSubRequests) {
requestChunk = requestList.subList(i, numOfSubRequests);
} else {
requestChunk = requestList.subList(i, i + MAX_BULK_SUBREQUEST_SIZE);
}
// parallelization
executorService.submit(() -> {
Response responseChunk = null;
try {
responseChunk = callService(requestChunk); // private method
} catch (XYZException e) {
...
try {
throw new Exception("Internal Server Error");
} catch (Exception ex) {
...
}
}
responseList.add(responseChunk);
});
}
executorService.shutdown();
try {
executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
} catch (InterruptedException e) {..}
}
return responseList;
}
private Response callService(..) {
// call to public method1
method1(..);
// call to public method2
method2(..);
}
I was able to do so with unit tests and adding a mockito verify on how many times a method is called. If it's running in parallel after chunkifying, then the method will be called more than once equal to number of chunks it's going to process.
I am trying to call a method multiple times every 60 seconds until a success response from the method which actually calls a rest end point on a different service. As of now I am using do while loop and using
Thread.sleep(60000);
to make the main thread wait 60 seconds which I feel is not the ideal way due to concurrency issues.
I came across the CountDownLatch method using
CountDownLatch latch = new CountDownLatch(1);
boolean processingCompleteWithin60Second = latch.await(60, TimeUnit.SECONDS);
#Override
public void run(){
String processStat = null;
try {
status = getStat(processStatId);
if("SUCCEEDED".equals(processStat))
{
latch.countDown();
}
} catch (Exception e) {
e.printStackTrace();
}
}
I have the run method in a different class which implements runnable. Not able to get this working. Any idea what is wrong?
You could use a CompletableFuture instead of CountDownLatch to return the result:
CompletableFuture<String> future = new CompletableFuture<>();
invokeYourLogicInAnotherThread(future);
String result = future.get(); // this blocks
And in another thread (possibly in a loop):
#Override
public void run() {
String processStat = null;
try {
status = getStat(processStatId);
if("SUCCEEDED".equals(processStat))
{
future.complete(processStat);
}
} catch (Exception e) {
future.completeExceptionally(e);
}
}
future.get() will block until something is submitted via complete() method and return the submitted value, or it will throw the exception supplied via completeExceptionally() wrapped in an ExecutionException.
There is also get() version with timeout limit:
String result = future.get(60, TimeUnit.SECONDS);
Finally got it to work using Executor Framework.
final int[] value = new int[1];
pollExecutor.scheduleWithFixedDelay(new Runnable() {
Map<String, String> statMap = null;
#Override
public void run() {
try {
statMap = coldService.doPoll(id);
} catch (Exception e) {
}
if (statMap != null) {
for (Map.Entry<String, String> entry : statMap
.entrySet()) {
if ("failed".equals(entry.getValue())) {
value[0] = 2;
pollExecutor.shutdown();
}
}
}
}
}, 0, 5, TimeUnit.MINUTES);
try {
pollExecutor.awaitTermination(40, TimeUnit.MINUTES);
} catch (InterruptedException e) {
}
I have a class called "Parser" that can be used to get a price from a url and parse it into an integer.
I then have other classes which uses those variables to create objects. Problem is that because it is running serially it is very slow.
How do I get them to parse the URL's in parallel?
public class Parser {
public static int getPrice(String url) {
String price = "";
try {
Document doc = Jsoup.connect(url).get();
price = doc.select("h3").select("span").attr("title");
} catch (IOException e) {
e.printStackTrace();
}
return parseInt(price);
}
public static double parseDouble(String parseMe) {
NumberFormat ukFormat = NumberFormat.getNumberInstance(Locale.UK);
double parsed = 0;
try {
parsed = ukFormat.parse(parseMe).doubleValue();
} catch (ParseException e) {
e.printStackTrace();
}
return parsed;
}
}
//Here is an example of the class
public class Example(){
private int field1, field2;
public Example(String url1, String url2){
field1=Parser.getPrice(url1);
field2=Parser.getPrice(url2);
}
}
If you'd like the getPrice calls to run asynchronously, you can use ExecutorService, like so:
public Example(String url1, String url2) {
// Create executorService.
ExecutorService executorService = Executors.newWorkStealingPool();
// Submit both tasks to executorService.
Future<Integer> future1 = executorService.submit(new Callable<Integer>() {
#Override
public Integer call() throws Exception {
return Parser.getPrice(url1);
}
});
Future<Integer> future2 = executorService.submit(new Callable<Integer>() {
#Override
public Integer call() throws Exception {
return Parser.getPrice(url2);
}
});
// Shutdown executorService. (It will no longer accept tasks, but will complete the ones in progress.)
executorService.shutdown();
// Handle results of the tasks.
try {
// Note: get() will block until the task is complete
field1 = future1.get();
field2 = future2.get();
} catch (InterruptedException e) {
// TODO Handle it
} catch (ExecutionException e) {
// TODO Handle it
}
}
I had exactly the same case, for me I had to had them to parse the two URL in the same function , and instead of returning an Integer, it returns instead an array of two integers, and it was more fast.
in your case I would suggest working with CyclicBarrier in a way that your code would look like:
final CyclicBarrier cb = new CyclicBarrier(2); // the parameter 2 is the number of threads that will invode the await method
long startTime = System.nanoTime();// this is just the start time to measure how many it took
Thread t1 = new Thread(){
public void run(){
try {
cb.await();
int field1 = Parser.getPrice(url1);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
e.printStackTrace();
}
}};
Thread t2 = new Thread(){
public void run(){
try {
cb.await();
int field2 = Parser.getPrice(url2);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
e.printStackTrace();
}
}};
t1.start();
t2.start();
long endTime = System.nanoTime();// end time of execution
long duration = (endTime - startTime);
System.out.println(duration);
I am trying to signal between two threads using the below FutureResult class which extends FutureTask class. When run the script, it prints the following result.
SENDING: 0
SENT: 0
POLL: FutureResult#513431
SIGNALLED: FutureResult#513431
Then the program hang up forever. I expect FutureResult instance should return the value from it's blocking get method. Then print the result in the console. But FutureResult.get is blocking forever.
import java.util.concurrent.*;
/**
* Created by someone on 20/08/2015.
*/
final public class FutureResult<T> extends FutureTask<T> {
private static final Object SS = "SS";
public FutureResult() {
super(() -> null);
}
public void signal(final T value) {
set(value);
}
public void signalError(final Throwable throwable) {
setException(throwable);
}
public static void main(String... args) throws Exception {
final ArrayBlockingQueue<FutureResult> queue = new ArrayBlockingQueue<>(1000000);
new Thread(() -> {
while (true) {
try {
final FutureResult poll = queue.take();
System.out.println("POLL: " + poll);
if (poll != null) {
poll.signal(SS);
System.out.println("SIGNALLED: " + poll);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}).start();
new Thread(() -> {
for (int i = 0; i < 1; i++) {
final FutureResult<Object> result = new FutureResult<>();
System.out.println("SENDING: " + i);
queue.offer(new FutureResult());
try {
System.out.println("SENT: " + i);
result.get();
System.out.println("GOT : " + i);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
}).start();
}
}
This is the problem:
queue.offer(new FutureResult());
You're setting the value on one FutureResult, but that's not the one you're waiting for. Just change that line to:
queue.offer(result);
and it works fine.
Looks like the confusion is in the use of FutureTask. FutureTask is designed as a Runnable; running it is necessary.
Honestly, based on the code, it looks like the custom code is implementing something similar to FutureTask. If the intent here is to learn to use FutureTask, then create a FutureTask instance with a "run" method, and then execute that run method. On completion of the run method, the FutureTask.get() will complete.
I'm writing an Android app that uses two threads. One is UI thread and the other handles server communication. Is it possible for the other thread to wait for a specified amount of time and then process all the messages that have arrived and then wait again?
I need this so that I can collect different data and send it to server in one session.
I've build my thread with HandlerThread but now I'm stuck. Can anyone point me to the right direction?
This is the code I'm using inside the second thread:
public synchronized void waitUntilReady() {
serverHandler = new Handler(getLooper()){
public void handleMessage(Message msg) { // msg queue
switch(msg.what) {
case TEST_MESSAGE:
testMessage(msg);
break;
case UI_MESSAGE:
break;
case SERVER_MESSAGE:
break;
default:
System.out.println(msg.obj != null ? msg.obj.getClass().getName() : "is null");
break;
}
}
};
}
EDIT:
I resolved my issue by going with Thread instead of HandlerThread and using queue.
I'm new to programming so I apologize for any horrenous errors but here's the code I ended up using.
public class ServiceThread extends Thread {
// TODO maybe set the thread priority to background?
static ServiceThread sThread = new ServiceThread(); // service thread instance
private volatile Handler mainHandler;
//
public Thread mainThread;
private boolean OK = true;
public Queue<MessageService> msgQueue;
private ThreadPoolExecutor exec;
private ServiceThread() { }
#Override
public void run() {
synchronized (this){
msgQueue = new ConcurrentLinkedQueue<MessageService>();
notifyAll();
}
mainHandler = new Handler(Looper.getMainLooper());
ThreadPoolExecutor exPool = (ThreadPoolExecutor) Executors.newFixedThreadPool(2);
exec = exPool;
// MAIN LOOP
try {
while(OK) {
getMessagesFromQueue();
Thread.sleep(3000);
}
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//end of loop
}
public void ProcessMessage(MessageService message) {
System.err.println("ProcessMessage with command: " + message.command);
}
/** Called from the Main thread. Waits until msgQueue is instantiated and then passes the reference
* #return Message Queue
*/
public Queue<MessageService> sendQueue() {
synchronized (this){
while(msgQueue == null) {
try {
wait();
} catch (InterruptedException e) {
// TODO Auto-generated catch block -- move the try block!
e.printStackTrace();
}
}
}
return msgQueue;
}
public void setOkFalse () {
if (OK == true)
OK = false;
}
// Message handling methods
/** Priority message from UI thread, processed in another thread ASAP.
* Should be used on commands like getBigPicture or getPics when cached pics are running out
* or upload picture etc.
* #param message - Message should always be MessageService class
* TODO check that it really is.
*/
public void prioTask (MessageService message) {
final MessageService taskMsg = message;
Runnable task = new Runnable() {
#Override
public void run(){
ProcessMessage(taskMsg);
}
};
exec.execute(task);
}
/**
* Gets messages from queue, puts them in the list, saves the number of messages retrieved
* and sends them to MessageService.handler(int commands, list messageList)
* (method parameters may change and probably will =) )
*/
public void getMessagesFromQueue() {
int commands = 0;
ArrayList <MessageService> msgList = new ArrayList <MessageService>();
while(!msgQueue.isEmpty()) {
if(msgQueue.peek() instanceof MessageService) {
//put into list?
msgList.add(msgQueue.remove());
commands++;
} else {
//Wrong type of message
msgQueue.remove();
System.err.println("getMessagesFromQueue: Message not" +
" instanceof MessageService, this shouldn't happen!");
}
}
if (commands > 0) {
HTTPConnection conn;
try {
conn = new HTTPConnection();
MessageService.handleSend(commands, msgList, conn);
conn.disconnect();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
P.S. This is also my first post here. Should I mark it solved or something? How?