We Keep Coding

Java ScheduledThreadPoolExecutor: Can Runnables in a List be executed after shutdownNow()?

I can't execute Runnables(delayed tasks in a queue), that have been returned in an list of Runnables after invoking of shutdownNow() on ScheduledThreadPoolExecutor object.
I've tried some ways to do it: you can get list size, one of the Runnable objects itself, invoke isDone() query, but I haven't coped to run them.
CAN they be executed and HOW if possible?
See please code below. Thank you.
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.FutureTask;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.ThreadPoolExecutor;
public class ExecuteExisitingDelayedTasksAfterShutdownPolicy1 {
private static int count = 0;
private static class Task implements Runnable {
private String name;
public Task(String name) {
this.name = name;
count++;
}
#Override
public void run() {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
return;
}
System.out.printf("\n%s: " + getName(), Thread.currentThread().getName());
}
public String getName() {
return name;
}
}
public static void main(String[] args) throws InterruptedException, ExecutionException {
ScheduledThreadPoolExecutor stpe = new ScheduledThreadPoolExecutor(10, new ThreadPoolExecutor.DiscardPolicy());
stpe.setExecuteExistingDelayedTasksAfterShutdownPolicy(true);
List<Runnable> queue = null;
for (int i = 0; i < 100; i++) {
stpe.execute(new Task("Task " + count));
if (i == 50) {
Thread.sleep(1000);
queue = stpe.shutdownNow();
System.out.print("\nQueue SIZE: " + queue.size());
}
}
Thread.sleep(3000);
System.out.print("\n" + queue.get(0));
#SuppressWarnings("unchecked")
FutureTask<Task> ftask = (FutureTask<Task>) queue.get(0);
ExecutorService ses = Executors.newSingleThreadExecutor();
/**
* all of the next.. doesn't work: tasks returned in a queue are likely
to be
* unrunnable
*/
ftask.get().run();
System.out.println(ftask.get().name);
ses.execute(ftask);
queue.get(0).run();
}
}

How to start a thread for an operation at each iteration of for loop?

I am using a for loop which run for 10 times and in each iteration driver move to different URL and loads a file which took 4 minutes to complete all 10 downloads. I was wondering if I could implement multi-threading in a for loop so that it starts a different thread for each iteration to execute the download process.
for(int i=1;i<=10;i++) {
WebDriver driver = new ChromeDriver(setChromePref(URL[i]));
obj_SjStrore = new SjStrore(driver);
driver.click.findelements(By.xpath("xpath string").click;
driver.close();
}

Something like this should work.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
public class ParallelDownload implements Runnable {
private String url;
private String xpath;
public ParallelDownload(String url, String xpath) {
super();
this.url = url;
this.xpath = xpath;
}
#Override
public void run() {
WebDriver driver = new ChromeDriver();
driver.get(url);
driver.findElement(By.xpath(xpath)).click();
driver.close();
}
public static void main(String[] args) {
String url[] = new String[10];
String xpath[] = new String[10];
// create thread executor with pool of 10 threads
ExecutorService executor = Executors.newFixedThreadPool(10);
for (int i = 0; i < 10; i++) {
// Initialize a thread and execute
Runnable worker = new ParallelDownload(url[i], xpath[i]);
executor.execute(worker);
}
executor.shutdown();
// wait till all the threads stops executing
while (!executor.isTerminated()) {
}
}
}

Retrofit 2.0 value from asynch thread

I am new into threads. And the question is: Is this correct way to extract values from async Retrofit thread. And why dont I get "0 code exit"?
Here is main code
import com.google.gson.GsonBuilder;
import iceandfire.Callbacks.RootCallback;
import iceandfire.Models.Root;
import iceandfire.Utils.IceFireClient;
import retrofit2.Call;
import retrofit2.Callback;
import retrofit2.Response;
import retrofit2.Retrofit;
import retrofit2.converter.gson.GsonConverterFactory;
public class Main implements RootCallback
{
public static void main(String [] args)
{
RequestRoot(new Main());
//this imitates UI thread
Thread thread = new Thread()
{
public void run()
{
int counter = 5;
while (counter >= 0)
{
System.out.println("I am not retrofit Call");
counter -= 1;
try
{
Thread.sleep(1000); // 1 second
} catch (Exception e)
{
e.printStackTrace();
}
}
}
};
thread.start();
}
public static void RequestRoot(final RootCallback rootCallback)
{
Retrofit.Builder retroBuilder = new Retrofit.Builder()
.baseUrl("https://anapioficeandfire.com/")
.addConverterFactory(GsonConverterFactory.create());
Retrofit retrofit = retroBuilder.build();
IceFireClient client = retrofit.create(IceFireClient.class);
Call<Root> kall = client.root();
kall.enqueue(new Callback<Root>()
{
#Override
public void onResponse(Call<Root> call, Response<Root> response)
{
Root bloodyRoot = response.body();
rootCallback.onSuccess(bloodyRoot);
}
#Override
public void onFailure(Call<Root> call, Throwable t)
{
}
});
}
#Override
public void onSuccess(Root bloodyRoot)
{
System.out.println( new GsonBuilder().setPrettyPrinting().create().toJson(bloodyRoot).toString());
}
}
And callback
package iceandfire.Callbacks;
import iceandfire.Models.Root;
public interface RootCallback
{
void onSuccess(Root bloodyRoot);
}
What I expect is: 5x messages "I am not Retrofit call";Json that I get from iceandfireapi/root at any random time before, after, or between previous messages. And the last but not least ending of everything:"Process finished with exit code 0"
But what I get is an infinite loop: 5x messages, Json and NO exit message.
P.S. Timer thread is used to "imitate" some UI things. While retrofit is doing requests.

Why is this code executing sequentially?

Below code :
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class ThreadTest {
private static int counter = 0;
private static ExecutorService executorService = Executors.newCachedThreadPool();
private static List<Integer> intValues = new ArrayList<Integer>();
public static void main(String args[]){
for(int counter = 0; counter < 10; ++counter){
intValues.add(testCallback());
}
for(int i : intValues){
System.out.println(i);
}
System.exit(0);
}
public static Integer testCallback() {
Future<Integer> result = executorService.submit(new Callable<Integer>() {
public Integer call() throws Exception {
counter += 1;
Thread.sleep(500);
return counter;
}
});
try {
return result.get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
return null;
}
}
Outputs :
1
2
3
4
5
6
7
8
9
10
This program takes approx 5 seconds to run. I am trying to execute multiple invocations of testCallback method in a seperate thread so I would expect this method to run in 10 threads concurrently where each thread uses approx 500 miliseconds of time. So over all I expet the program to run in < 1 second.
Why is counter not being invoked in seperate threads concurrently ?

result.get();
This is a blocking call that waits for the task to complete.
Therefore, you're waiting for each task to finish before starting the next one.

HtmlUnit WebClient Timeout

In my previous questions about HtmlUnit
Skip particular Javascript execution in HTML unit
and
Fetch Page source using HtmlUnit : URL got stuck
I had mentioned that URL is getting stuck. I also found out that it is getting stuck due to one of the methods(parse) in HtmlUnit library is not coming out of execution.
I did further work on this. I wrote code to get out of the method if it takes more than specified time-out seconds to complete.
import java.io.IOException;
import java.net.MalformedURLException;
import java.util.Date;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
public class HandleHtmlUnitTimeout {
public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException, InterruptedException, TimeoutException
{
Date start = new Date();
String url = "http://ericaweiner.com/collections/";
doWorkWithTimeout(url, 60);
}
public static void doWorkWithTimeout(final String url, long timeoutSecs) throws InterruptedException, TimeoutException {
//maintains a thread for executing the doWork method
ExecutorService executor = Executors.newFixedThreadPool(1);
//logger.info("Starting method with "+timeoutSecs+" seconds as timeout");
//set the executor thread working
final Future<?> future = executor.submit(new Runnable() {
public void run()
{
try
{
getPageSource(url);
}
catch (Exception e)
{
throw new RuntimeException(e);
}
}
});
//check the outcome of the executor thread and limit the time allowed for it to complete
try {
future.get(timeoutSecs, TimeUnit.SECONDS);
} catch (Exception e) {
//ExecutionException: deliverer threw exception
//TimeoutException: didn't complete within downloadTimeoutSecs
//InterruptedException: the executor thread was interrupted
//interrupts the worker thread if necessary
future.cancel(true);
//logger.warn("encountered problem while doing some work", e);
throw new TimeoutException();
}finally{
executor.shutdownNow();
}
}
public static void getPageSource(String productPageUrl)
{
try {
if(productPageUrl == null)
{
productPageUrl = "http://ericaweiner.com/collections/";
}
WebClient wb = new WebClient(BrowserVersion.FIREFOX_3_6);
wb.getOptions().setTimeout(120000);
wb.getOptions().setJavaScriptEnabled(true);
wb.getOptions().setThrowExceptionOnScriptError(true);
wb.getOptions().setThrowExceptionOnFailingStatusCode(false);
HtmlPage page = wb.getPage(productPageUrl);
wb.waitForBackgroundJavaScript(4000);
wb.closeAllWindows();
}
catch (FailingHttpStatusCodeException e)
{
e.printStackTrace();
}
catch (MalformedURLException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
This code does come out of doWorkWithTimeout(url, 60); method. But this does not terminate.
When I try to call similiar implementation with following code:
import java.util.Date;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import org.apache.log4j.Logger;
public class HandleScraperTimeOut {
private static Logger logger = Logger.getLogger(HandleScraperTimeOut .class);
public void doWork() throws InterruptedException {
logger.info(new Date()+ "Starting worker method ");
Thread.sleep(20000);
logger.info(new Date()+ "Ending worker method ");
//perform some long running task here...
}
public void doWorkWithTimeout(int timeoutSecs) {
//maintains a thread for executing the doWork method
ExecutorService executor = Executors.newFixedThreadPool(1);
logger.info("Starting method with "+timeoutSecs+" seconds as timeout");
//set the executor thread working
final Future<?> future = executor.submit(new Runnable() {
public void run()
{
try
{
doWork();
}
catch (Exception e)
{
throw new RuntimeException(e);
}
}
});
//check the outcome of the executor thread and limit the time allowed for it to complete
try {
future.get(timeoutSecs, TimeUnit.SECONDS);
} catch (Exception e) {
//ExecutionException: deliverer threw exception
//TimeoutException: didn't complete within downloadTimeoutSecs
//InterruptedException: the executor thread was interrupted
//interrupts the worker thread if necessary
future.cancel(true);
logger.warn("encountered problem while doing some work", e);
}
executor.shutdown();
}
public static void main(String a[])
{
HandleScraperTimeOut hcto = new HandleScraperTimeOut ();
hcto.doWorkWithTimeout(30);
}
}
If anybody can have a look and tell me what is the issue, it will be really helpful.
For more details about issue, you can look into Skip particular Javascript execution in HTML unit
and
Fetch Page source using HtmlUnit : URL got stuck
Update 1
Strange thing is : future.cancel(true); is returning TRUE in both cases.
How I expected it to be was :
With HtmlUnit it should return FALSE since process is still hanging.
With normal Thread.sleep(); it should return TRUE since the process
got cancelled successfully.
Update 2
It only hangs with http://ericaweiner.com/collections/ URL. If I give any other URL i.e. http://www.google.com , http://www.yahoo.com , It does not hand. In these cases it throws IntruptedException and come out of the Process.
It seems that http://ericaweiner.com/collections/ page source has certain elements which are causing problems.

Future.cancel(boolean) returns:
false if the task could not be cancelled, typically because it has already completed normally
true otherwise
Cancelled means means the thread did not finish before cancel, the canceled flag was set to true and if requested the thread was interrupted.
Interrupt the thread menans it called Thread.interrupt and nothing more. Future.cancel(boolean) does not check if the thread actually stopped.
So it is right that cancel return true on that cases.
Interrupting a thread means it should stop as soon as possible but it is not enforced. You can try to make it stop/fail closing a resource it needs or something. I usually do that with a thread reading (waiting incoming data) from a socket. I close the socket so it stops waiting.