I have a class called "Parser" that can be used to get a price from a url and parse it into an integer.
I then have other classes which uses those variables to create objects. Problem is that because it is running serially it is very slow.
How do I get them to parse the URL's in parallel?
public class Parser {
public static int getPrice(String url) {
String price = "";
try {
Document doc = Jsoup.connect(url).get();
price = doc.select("h3").select("span").attr("title");
} catch (IOException e) {
e.printStackTrace();
}
return parseInt(price);
}
public static double parseDouble(String parseMe) {
NumberFormat ukFormat = NumberFormat.getNumberInstance(Locale.UK);
double parsed = 0;
try {
parsed = ukFormat.parse(parseMe).doubleValue();
} catch (ParseException e) {
e.printStackTrace();
}
return parsed;
}
}
//Here is an example of the class
public class Example(){
private int field1, field2;
public Example(String url1, String url2){
field1=Parser.getPrice(url1);
field2=Parser.getPrice(url2);
}
}
If you'd like the getPrice calls to run asynchronously, you can use ExecutorService, like so:
public Example(String url1, String url2) {
// Create executorService.
ExecutorService executorService = Executors.newWorkStealingPool();
// Submit both tasks to executorService.
Future<Integer> future1 = executorService.submit(new Callable<Integer>() {
#Override
public Integer call() throws Exception {
return Parser.getPrice(url1);
}
});
Future<Integer> future2 = executorService.submit(new Callable<Integer>() {
#Override
public Integer call() throws Exception {
return Parser.getPrice(url2);
}
});
// Shutdown executorService. (It will no longer accept tasks, but will complete the ones in progress.)
executorService.shutdown();
// Handle results of the tasks.
try {
// Note: get() will block until the task is complete
field1 = future1.get();
field2 = future2.get();
} catch (InterruptedException e) {
// TODO Handle it
} catch (ExecutionException e) {
// TODO Handle it
}
}
I had exactly the same case, for me I had to had them to parse the two URL in the same function , and instead of returning an Integer, it returns instead an array of two integers, and it was more fast.
in your case I would suggest working with CyclicBarrier in a way that your code would look like:
final CyclicBarrier cb = new CyclicBarrier(2); // the parameter 2 is the number of threads that will invode the await method
long startTime = System.nanoTime();// this is just the start time to measure how many it took
Thread t1 = new Thread(){
public void run(){
try {
cb.await();
int field1 = Parser.getPrice(url1);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
e.printStackTrace();
}
}};
Thread t2 = new Thread(){
public void run(){
try {
cb.await();
int field2 = Parser.getPrice(url2);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
e.printStackTrace();
}
}};
t1.start();
t2.start();
long endTime = System.nanoTime();// end time of execution
long duration = (endTime - startTime);
System.out.println(duration);
Related
I am trying to call a method multiple times every 60 seconds until a success response from the method which actually calls a rest end point on a different service. As of now I am using do while loop and using
Thread.sleep(60000);
to make the main thread wait 60 seconds which I feel is not the ideal way due to concurrency issues.
I came across the CountDownLatch method using
CountDownLatch latch = new CountDownLatch(1);
boolean processingCompleteWithin60Second = latch.await(60, TimeUnit.SECONDS);
#Override
public void run(){
String processStat = null;
try {
status = getStat(processStatId);
if("SUCCEEDED".equals(processStat))
{
latch.countDown();
}
} catch (Exception e) {
e.printStackTrace();
}
}
I have the run method in a different class which implements runnable. Not able to get this working. Any idea what is wrong?
You could use a CompletableFuture instead of CountDownLatch to return the result:
CompletableFuture<String> future = new CompletableFuture<>();
invokeYourLogicInAnotherThread(future);
String result = future.get(); // this blocks
And in another thread (possibly in a loop):
#Override
public void run() {
String processStat = null;
try {
status = getStat(processStatId);
if("SUCCEEDED".equals(processStat))
{
future.complete(processStat);
}
} catch (Exception e) {
future.completeExceptionally(e);
}
}
future.get() will block until something is submitted via complete() method and return the submitted value, or it will throw the exception supplied via completeExceptionally() wrapped in an ExecutionException.
There is also get() version with timeout limit:
String result = future.get(60, TimeUnit.SECONDS);
Finally got it to work using Executor Framework.
final int[] value = new int[1];
pollExecutor.scheduleWithFixedDelay(new Runnable() {
Map<String, String> statMap = null;
#Override
public void run() {
try {
statMap = coldService.doPoll(id);
} catch (Exception e) {
}
if (statMap != null) {
for (Map.Entry<String, String> entry : statMap
.entrySet()) {
if ("failed".equals(entry.getValue())) {
value[0] = 2;
pollExecutor.shutdown();
}
}
}
}
}, 0, 5, TimeUnit.MINUTES);
try {
pollExecutor.awaitTermination(40, TimeUnit.MINUTES);
} catch (InterruptedException e) {
}
i'm trying create a thread, which return a value, the process is running correctly but my screen is still locked. I want a thread that return a value but my main thread continues running.
I've done that:
public void showPartidas(int maximumDistance){
ExecutorService es = Executors.newFixedThreadPool(1);
Future<ArrayList<Partida>> partidas= es.submit(new FilterPartidas(maximumDistance));
try {
loadInListView(partidas.get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
es.shutdown();
}
class FilterPartidas implements Callable<ArrayList<Partida> > {
private final int distance;
private ArrayList<Partida> partidas;
FilterPartidas(int distance) {
this.distance = distance;
}
#Override
public ArrayList<Partida> call() throws Exception {
partidas=null;
Download load = new Download();
Date fecha = new Date();
DateFormat fechaFormat = new SimpleDateFormat("yyyy-MM-dd");
String query = "select * from partidas where fecha >='"+fechaFormat.format(fecha)+"'";
partidas=load.obtainPartidas(query, distance, myPosition);
return partidas;
}
}
partidas.get() action is the cause that main thread is waiting for the completion of Callable method in executor. If you want main thread are still running during Callable action execution you must place partidas.get() action into dedicated separate thread e.g.:
replace
Future<ArrayList<Partida>> partidas= es.submit(new FilterPartidas(maximumDistance));
try {
loadInListView(partidas.get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
into
final Future<ArrayList<Partida>> partidas= es.submit(new FilterPartidas(maximumDistance));
new Thread(new Runnable() {
#Override
public void run() {
try {
loadInListView(partidas.get());
} catch (InterruptedEArrayList<Partida>xception e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
}).start();
or similar action with threads (maybe using executor, Runnable, etc).
Or you can change you logic (if possible) and hide call to method from Callable into Runnable class. E,g.:
ExecutorService es = Executors.newFixedThreadPool(1);
es.submit(new Runnable() {
#Override
public void run() {
ArrayList<Partida> partidas = logic from you Callable call;
loadInListView(partidas);
}
});
This question already has answers here:
How to pass a function as a parameter in Java? [duplicate]
(8 answers)
Closed 8 years ago.
I'm working on a java webserver which calls a web service. I'm trying to write some code in a nicer.
I've got 2 similar functions, only one thing changes : the function i have to call depending what i want to do.
public Object updateDef() {
ExecutorService pool = Executors.newFixedThreadPool(20);
List<Callable<String>> tasks = new ArrayList<Callable<String>>();
logger.info("Updating surv def...");
for (final Surv surv : Client.activatedSurvs) {
final Client client = new Client(surv.getName(), surv.getPassword());
tasks.add(new Callable<String>() {
public String call() throws Exception {
HERE -> client.updateDef(surv);
return surv.getId();
}
});
client.destroy();
}
List<Future<String>> results = null;
try {
results = pool.invokeAll(tasks);
for (Future<String> future : results) {
future.get();
}
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
pool.shutdown();
return null;
}
And the other one :
public Object updateStats() {
ExecutorService pool = Executors.newFixedThreadPool(20);
List<Callable<String>> tasks = new ArrayList<Callable<String>>();
logger.info("Updating surv Stats...");
for (final Surv surv : Client.activatedSurvs) {
final Client client = new Client(surv.getName(), surv.getPassword());
tasks.add(new Callable<String>() {
public String call() throws Exception {
HERE -> client.updateStats(surv).toString();
return surv.getId();
}
});
client.destroy();
}
List<Future<String>> results = null;
try {
results = pool.invokeAll(tasks);
for (Future<String> future : results) {
future.get();
}
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
pool.shutdown();
return null;
}
So the 2 functions are updateStats and updateDef from class A with a static list from B.
The problem is that I've to pass a parameter which is an existing element from this list.
=> A.updateStats(B.list.get(X));
=> A.updateDef(B.list.get(X));
I visited this link How to pass a function as a parameter in Java? but without success
Any idea ?
Your object is to replace:
tasks.add(new Callable<String>() {
public String call() throws Exception {
client.updateStats(surv).toString();
return surv.getId();
}
});
With
tasks.add(new Callable<String>() {
public String call() throws Exception {
doSomething(client, surv);
return surv.getId();
}
});
Which can be done by defining an Interface:
public interface ClientUpdate {
public void performUpdate(Client client, Surv surv);
}
Then, use this interface as a parameter to your method:
public Object updateStats() {
Becomes
public Object update(ClientUpdate updater) {
Then use it in your callable
tasks.add(new Callable<String>() {
public String call() throws Exception {
updater.performUpdate(client, surv);
return surv.getId();
}
});
Then implement it:
updateStats();
becomes:
update(new ClientUpdate() {
public void performUpdate(Client client, Surv surv) {
client.updateStats(surv).toString();
}
});
Similarly for updateDef();
You could use a Strategy pattern or Functors. Honestly, I think I would just factor the common pieces out into shared methods.
Easy - pass in the Callable and don't repeat all the boilerplate around it.
public Object foo(Callable<String> callable) {
ExecutorService pool = Executors.newFixedThreadPool(20);
List<Callable<String>> tasks = new ArrayList<Callable<String>>();
logger.info("Updating surv def...");
for (final Surv surv : Client.activatedSurvs) {
final Client client = new Client(surv.getName(), surv.getPassword());
tasks.add(callable;
client.destroy();
}
List<Future<String>> results = null;
try {
results = pool.invokeAll(tasks);
for (Future<String> future : results) {
future.get();
}
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
pool.shutdown();
return null;
}
I'm learning concurrency and made some naive program to play with ExecutorService and Future tasks.
Also I want to check why instanceof is bad in some cases.
public class Test {
static enum Some {
FOO;
}
static abstract class Foo {
public abstract Some getType();
}
static class FooExt extends Foo {
public Some getType() {
return Some.FOO;
}
}
public static void main(String[] args) {
ExecutorService service = Executors.newFixedThreadPool(2);
final CountDownLatch start = new CountDownLatch(1);
Future<Integer> f1 = service.submit(new Callable<Integer>() {
#Override
public Integer call() {
try {
start.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Task started...");
int a = 0;
Foo foo = new FooExt();
while (!Thread.currentThread().isInterrupted()) {
if (foo instanceof FooExt) {
a++;
}
}
System.out.println("Task ended...");
return a;
}
});
Future<Integer> f2 = service.submit(new Callable<Integer>() {
#Override
public Integer call() {
try {
start.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Task started...");
int a = 0;
Foo foo = new FooExt();
while (!Thread.currentThread().isInterrupted()) {
if (foo.getType() == Some.FOO) {
a++;
}
}
System.out.println("Task ended...");
return a;
}
});
start.countDown();
try {
Thread.sleep(100);
} catch (InterruptedException e) {
}
service.shutdownNow();
System.out.println("service is shutdowned...");
try {
System.out.println("instanceof: "+f1.get());
System.out.println("enum: "+f2.get());
} catch (Exception e) {
e.printStackTrace();
}
}
}
but unfortunately my code is never terminated, and I cant get any values from my tasks :(
Hi I have executed your program. I got the following output:
Task started...
Task started...
Task ended...
service is shutdowned...
Task ended...
instanceof: 1287184
enum: 1247375
This code terminates.
I have to call more than one webservice in one method each webservice is executed by separate thread in concurrent/parellel. Every web service will return one ArrayList. Note: may chance some webservices will fail or take more time process response in this case i have to skip these failure result. How can I achieve this? I tried this sample code.
public class MultiThreadsEx{
public class Task implements Runnable {
private Object result;
private String id;
int maxRowCount = 0;
public Task(String id) {
this.id = id;
}
public Object getResult() {
return result;
}
public void run() {
try {
System.out.println("Running id=" + id+" at "+Utilities.getCurrentJavaDate("DD/MM/YYYY HH:MM:SS"));
if(id.equalsIgnoreCase("1")){
/**Getting Details from Amazon WS*/
maxRowCount = AmazonUtils.getweather(cityname);
}else if(id.equalsIgnoreCase("2")){
/**Getting Details from Google WS* /
maxRowCount = GoogleUtils.getWeather(cityName);
}
// call web service
//Thread.sleep(1000);
//result = id + " more";
result = maxRowCount;
} catch (InterruptedException e) {
// TODO do something with the error
throw new RuntimeException("caught InterruptedException", e);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public static void runInParallel(Runnable runnable1, Runnable runnable2) {
try {
Thread t1 = new Thread(runnable1);
Thread t2 = new Thread(runnable2);
t1.start();
t2.start();
} catch (Exception e) {
// TODO do something nice with exception
throw new RuntimeException("caught InterruptedException", e);
}
}
public void foo() {
try {
Task task1 = new Task("1");
Task task2 = new Task("2");
runInParallel(task1, task2);
System.out.println("task1 = " + task1.getResult()+" at "+Utilities.getCurrentJavaDate("DD/MM/YYYY HH:MM:SS"));
System.out.println("task2 = " + task2.getResult()+" at "+Utilities.getCurrentJavaDate("DD/MM/YYYY HH:MM:SS"));
} catch (Exception e) {
//TODO Auto-generated catch block
e.printStackTrace();
}
}
But run() return type is void so how can return result? Examples are highly appreciated. I am new to multithread/concurrent threads concept so if I'm doing anything wrong, please point me in the right direction.
Consider replacing Runnable - run with Callable - call. This will allow you to return a result from your thread task:
public class Task implements Callable<Object> {
private Object result;
public Object call() {
// compute result
return result;
}
}
Now use an ExecutorService:
public static void runInParallel(Callable<Object> c1, Callable<Object> c2) {
ExecutorService exec = Executors.newFixedThreadPool(2);
Future<Object> f1 = exec.submit(c1);
Future<Object> f2 = exec.submit(c2);
}
Later in the code you can use f1.get() and f2.get() to wait for the results of the tasks.
The usual way to communicate the results of a Runnable back to the object which created it is by passing the creating object to the constructor of the Runnable. When the task is finished, you can call a method in the creating object and pass the result data.