We Keep Coding

Resubmit Callable to executorService on exception

My situation
I'm trying to craft a functionality which would execute n (where n >=0) requests to a given endpoint, but I do understand that sometimes that endpoint might not respond due to
500 error or other issue, so I want to repeat my requests to an endpoint (with a
small interval in between [not yet implemented]) till I get a response, or till I get an unknown error which would indicate what I can't repeat, because of other reasons than a crashed server.
So, I've tried to implement this piece of functionality using Executors and concurrency provided by Java 11 and it does not work as I want
I can't resubmit failed tasks till I get all the responses and I don't know why
I have a method
private void DoMyTasks(List<MyRequest> requests) {
final ExecutorService executorService = Executors.newFixedThreadPool(10);
final ExecutorCompletionService<MyReqResDto> completionService =
new ExecutorCompletionService<>(executorService);
for (final MyRequest MyRequest : requests) {
completionService.submit(new MyCallableRequest(webClient, MyRequest));
}
List<MyReqResDto> responses = new ArrayList<>();
for (int i = 0; i < requests.size(); ++i) {
try {
final Future<MyReqResDto> future = completionService.take();
if (future.get().getEx() != null) {
completionService.submit(new MyCallableRequest(webClient, future.get().getMyRequest()));
}
responses.add(future.get());
} catch (ExecutionException | InterruptedException e) {
log.warn("Error"));
} catch (Exception exception) {
log.error("Other error");
} finally {
executorService.shutdown();
try {
if (!executorService.awaitTermination(10, TimeUnit.MINUTES)) {
executorService.shutdownNow();
}
} catch (InterruptedException e) {
executorService.shutdownNow();
}
}
}
responses.size();
}
I'm trying to repeat failed tasks with
if (future.get().getEx() != null) {
completionService.submit(new MyCallableRequest(webClient, future.get().getMyRequest()));
}
and yet, at the end of execution I don't get all responses for my requests. What I get is at most 3 to 5 responses when I try executing 10 requests. Why? How to fix it?
My callable class is
public class MyCallableRequest implements Callable<MyReqResDto> {
private final WebClient webClient;
private final MyRequest myRequest;
public MyCallableRequest(WebClient webClient, MyRequest myRequest) {
this.webClient = webClient;
this.myRequest = myRequest;
}
#Override
public MyReqResDto call() throws Exception {
try {
if (new Random().nextInt(10) % 2 == 0) {
throw new TestException();
}
if (new Random().nextInt(10) % 7 == 0) {
throw new RuntimeException();
}
WebClient.UriSpec<WebClient.RequestBodySpec> uriSpec = webClient.post();
WebClient.RequestBodySpec bodySpec = uriSpec.uri(
s -> s.path("/myEndpoint").build());
MyRequestDto myMyRequestDto = new MyRequestDto();
WebClient.RequestHeadersSpec<?> headersSpec =
bodySpec.body(Mono.just(myMyRequestDto), MyRequestDto.class);
ResponseDto responseDto = headersSpec.exchangeToMono(s -> {
if (s.statusCode().equals(HttpStatus.OK)) {
return s.bodyToMono(ResponseDto.class);
} else if (s.statusCode().is1xxInformational()) {
return s.createException().flatMap(Mono::error);
} else if (s.statusCode().is3xxRedirection()) {
return s.createException().flatMap(Mono::error);
} else if (s.statusCode().is4xxClientError()) {
return s.createException().flatMap(Mono::error);
} else if (s.statusCode().is5xxServerError()) {
return s.createException().flatMap(Mono::error);
} else {
return s.createException().flatMap(Mono::error);
}
//return null;
}).block();
return new MyReqResDto(myRequest, responseDto, null);
} catch (Exception exception) {
return new MyReqResDto(myRequest, null, exception);
}
}
}
Update NO. 1
I changed a for loop to a while loop according to a comment provided by
Slaw and an answer provided by erickson. And this solutions works, meaning that
it is hammering an endpoint till all responses are received without
any errors. But I'm still not sure it feels that I'm building a sh**
tower with this solution. Is there any thread related issues that I should be aware while using executor like this?
while (true) {
Future < MyReqResDto > future = null;
try {
future = completionService.take();
if (future.get().getEx() != null /*and check exception if possible to handle, if not break from a loop*/) {
completionService.submit(new MyCallableRequest(webClient, future.get().getRequestCT());
} else {
responseDtos.add(future.get());
}
} catch (ExecutionException | InterruptedException e) {
log.warn("Error while downloading", e.getCause());
// test if I can recover from these exceptions if no
break;
}
}
if (responseDtos.size() == requests.size()) {
executorService.shutdown();
try {
if (!executorService.awaitTermination(10, TimeUnit.MINUTES)) {
executorService.shutdownNow();
}
} catch (InterruptedException e) {
executorService.shutdownNow();
}
break;
}

You are shutting down the executor as soon as you get one response. Perhaps a few more have completed in this time, but you are not allowing time for any others to complete.
Your logic here is wrong. The executor should only be shut down when you are sure no more tasks will be submitted; at soonest, that is after the loop responsible for re-submitting failures.
Here is a simplified view of your code to highlight the premature shutdown:
for (int i = 0; i < requests.size(); ++i) {
try {
final Future<MyReqResDto> future = completionService.take();
...
responses.add(future.get());
...
} finally {
executorService.shutdown();
}
}

JDI ThreadReference.frame() causes IncompatibleThreadStateException

Currently I am trying to extract some execution data via the JDI.
Therefore I first start a java vm manually with the command
java -agentlib:jdwp=transport=dt_socket,server=y,address=8000 DebugDummy
My DebugDummy.java:
public class DebugDummy {
public class MyInnerClass {
private int a;
public MyInnerClass(int a) {
this.a = a;
this.doSomething();
}
public void doSomething() {
System.out.println(this.a);
}
}
public DebugDummy() {
MyInnerClass myInnerClass = new MyInnerClass(5);
myInnerClass.doSomething();
}
public static void main(String[] args) {
DebugDummy dd = new DebugDummy();
}
}
And then connect with the JDI, waiting for the main-method entry and step line per line through the code execution.
public class VMStart {
public static void main(String[] args) {
//Check for argument count
if(args.length != 3){
System.err.println("Not enough parameter!");
System.exit(0);
}
String cwd = "", mainClass = "", vmPort = "";
cwd = args[0];
mainClass = args[1];
vmPort = args[2];
System.out.println("CWD: " + cwd);
System.out.println("MainClass: " + mainClass);
System.out.println("VM Port: " + vmPort);
//Init vm arguments and settings
VirtualMachineManager vmm = Bootstrap.virtualMachineManager();
AttachingConnector ac = vmm.attachingConnectors().get(0);
//Setting port
Map<String, Connector.Argument> env = ac.defaultArguments();
Connector.Argument port = env.get("port");
port.setValue(vmPort);
//Setting hostname
Connector.Argument hostname = env.get("hostname");
hostname.setValue("localhost");
//Attach vm to remote vm
VirtualMachine vm = null;
try {
vm = ac.attach(env);
} catch (IOException | IllegalConnectorArgumentsException e) {
//Doesn't work, stop here...
System.err.println("Can't connect to vm!");
e.printStackTrace();
System.exit(0);
}
//Create EventQueue and EventRequestManager for further event handling
EventQueue eventQueue = vm.eventQueue();
EventRequestManager mgr = vm.eventRequestManager();
//Set the vm to sleep for further operations
vm.suspend();
//Searching for our main thread reference
ThreadReference mainThread = null;
List<ThreadReference> threads = vm.allThreads();
for (ThreadReference thread : threads) {
if ("main".equals(thread.name())) {
mainThread = thread;
}
}
//Create and register MethodEntryRequest, so we can pause execution at first line of main later on
MethodEntryRequest methodEntryRequest = mgr.createMethodEntryRequest();
methodEntryRequest.addClassFilter(mainClass);
methodEntryRequest.addThreadFilter(mainThread);
methodEntryRequest.enable();
//Resume the execution of the remote vm
vm.resume();
//Resume the execution of the main thread in remote vm
mainThread.resume();
//Waiting for our needed MethodEntryEvent so execution started at first line of main method
Event event = null;
while (true) {
EventSet eventSet = null;
try {
eventSet = eventQueue.remove();
} catch (InterruptedException e) {
System.err.println("Something went wrong while waiting for MethodEntryEvent");
e.printStackTrace();
System.exit(0);
}
event = eventSet.eventIterator().next();
if (event instanceof MethodEntryEvent) {
break;
}
}
//Indicates whether there is still code execution in our remote vm
boolean codeIsExecuting = true;
//Step loop until there is no code execution
while(codeIsExecuting){
//Filter for steeping not into java api methods
final String[] noBreakpointRequests = {"java.*", "javax.*", "sun.*", "com.sun.*"};
//Creating our StepRequest for each line of code
StepRequest stepRequest = mgr.createStepRequest(mainThread, StepRequest.STEP_LINE, StepRequest.STEP_INTO);
for(String classInFilter : noBreakpointRequests){ //Apply filter
stepRequest.addClassExclusionFilter(classInFilter);
}
stepRequest.addCountFilter(1);
try{
stepRequest.enable();
}catch(IllegalThreadStateException e){
//program reached end of code, so there is no code execution anymore
codeIsExecuting = false;
System.out.println("Code execution ended...");
break;
}
//Extract data from current vm execution state
//TODO
System.out.println("TODO - extract data from current vm execution state");
//Test
StackFrame stackFrame = null;
try {
stackFrame = mainThread.frame(0);
System.out.println(stackFrame.location());
} catch (IncompatibleThreadStateException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//Resume vm for code execution
vm.resume();
try {
Thread.sleep(10L);
} catch (InterruptedException e) {
e.printStackTrace();
}
//StepRequest is done, remove it from event queue to prevent lock
mgr.deleteEventRequest(stepRequest);
}
//Debugging has ended, we free our remote vm
try{
vm.dispose();
}catch(VMDisconnectedException e){
}
//Print results
//TODO
System.out.println("TODO - print results...");
}
}
Unfortunally, if I delete the Thread.sleep(), I'm getting a IncompatibleThreadStateException at line
stackFrame = mainThread.frame(0);
Exception:
com.sun.jdi.IncompatibleThreadStateException at com.sun.tools.jdi.ThreadReferenceImpl.privateFrames(ThreadReferenceImpl.java:436)
at com.sun.tools.jdi.ThreadReferenceImpl.frame(ThreadReferenceImpl.java:355)
at VMStart.main(VMStart.java:143)
What's wrong? Do I have to wait for a certain event before stepping one line further?

Finally, I found a solution to myself...
This code has to be inserted AFTER the vm.resume() instruction:
boolean go = false;
while (!go) {
EventSet eventSet = null;
try {
eventSet = eventQueue.remove();
} catch (InterruptedException e) {
System.err.println("Something went wrong while waiting for StepEvent");
e.printStackTrace();
System.exit(0);
}
for (Event e : eventSet) {
if (e instanceof StepEvent) {
System.out.println("Step Event!");
go = true;
}
}
}

SwingWorker issue, method doInBackground not executed?

Sometimes the doInBackgorund() method of my SwingWorker seems not to be executed, it goes directly to the done() method without saving or printing anything on some of my clients machines, so i suppose it's a random thing , and i can't figure out why. Here 's my code :
public class saveCmdWorker extends SwingWorker<Integer, Integer> {
Order ord;
public saveCmdWorker(Order ord) {
this.ord = ord;
}
#Override
public Integer doInBackground() {
if(999 != ord.getCaissier().getIdCaissier())
saveCmd(ord); // database queries
else
JOptionPane.showMessageDialog(null,"Error",JOptionPane.WARNING_MESSAGE);
if(ord.isIsProd() == false){
try {
// print via serial port
Printer.print(ord, false, Restaurant.numCaisse);
} catch (Exception ex) {
PosO2.errorLogger.log(Level.SEVERE, "Printing error", ex);
}
}
try {
Printer.printFacture(ord, false);
if(btnDuplicata.getForeground() == Color.red)
Printer.printFacture(ord, true);
} catch (Exception ex) {
PosO2.errorLogger.log(Level.SEVERE, "Printing error", ex);
}
return 1;
}
#Override
protected void done() {
try {
btnDuplicata.setForeground(Color.black);
ARendre = 0.0;
ord.clear();
for (int j = 0; j < tab_paiement.size(); j++) {
tab_paiement.get(j).setVisible(true);
}
montantRestant.setBackground(Color.red);
} catch(Exception e) {
PosO2.errorLogger.log(Level.SEVERE, "Refresh Error", e);
}
}
}
I execute this worker via this actionlistener :
ActionListener encaissListener = new ActionListener() {
#Override
public void actionPerformed(ActionEvent e) {
worker = new saveCmdWorker(cmd);
worker.execute();
}
};
I don't have any logs available so i assume no exception is caught. I saw that a JOptionPane was fired in the doInBackground()(consider as ui modification in an other thread?) but the problem exists when the application doesn't go in the else statement. Can this be the cause of my problems? I don't have this bug on my computer, it just works fine.

As per the SwingWorker documentation (http://docs.oracle.com/javase/7/docs/api/javax/swing/SwingWorker.html#execute()):
SwingWorker is only designed to be executed once. Executing a SwingWorker more than once will not result in invoking the doInBackground method twice.
So, it looks like you need to create a new instance of your subclass each time you want to run the execute method properly.

Is there a cleaner way to write this polling loop?

I am writing automated test cases in Selenium/WebDriver in java. I have the following code implemented to poll for existing WebElements, but as I am not an expert in Java I was wondering if there is a cleaner way to write this method:
/** selects Business index type from add split button */
protected void selectBusinessLink() throws Exception
{
Calendar rightNow = Calendar.getInstance();
Calendar stopPolling = rightNow;
stopPolling.add(Calendar.SECOND, 30);
WebElement businessLink = null;
while (!Calendar.getInstance().after(stopPolling))
{
try
{
businessLink = findElementByLinkText("Business");
businessLink.click();
break;
}
catch (StaleElementReferenceException e)
{
Thread.sleep(100);
}
catch (NoSuchElementException e)
{
Thread.sleep(100);
}
catch (ElementNotVisibleException e)
{
Thread.sleep(100);
}
}
if (businessLink == null)
{
throw new SystemException("Could not find Business Link");
}
}
This particular line is what makes me think the code is a little dirty:
while (!Calendar.getInstance().after(stopPolling))

You can do something like this
long t = System.currentMillis(); // actual time in milliseconds from Jan 1st 1970.
while (t > System.currentMillis() - 30000 ) {
...

How about using the System time in millis?
Calendar calendar = Calendar.getInstance();
calendar.add(Calendar.SECOND, 30);
long stopPollingTime = calendar.getTimeInMillis();
while (System.currentTimeMillis() < stopPollingTime) {
System.out.println("Polling");
try {
Thread.sleep(100);
} catch (InterruptedException e) {
}
}

How to make a Java thread hang

I am trying to create a solution to treat hung threads due to memory leaks, locked resources in our applications. One of the main problems I am having is trying to simulate a hung thread to deal with it. Any sugestions?
This is what I tried, but it just doesn't seem to do the job. Any thoughts?
class KillerThread extends Thread{
public KillerThread() {
super();
}
public KillerThread(String threadName) {
super(threadName);
}
public void run (){
System.out.println("Start of KillerThread " + this.getName() );
if ( System.currentTimeMillis() % 2L == 0 ){
try {
sleep(Long.MAX_VALUE);
} catch (InterruptedException e) {
e.printStackTrace();
}
} else {
for(;;);
}
}
}

Joining on one's own thread works well for me:
Thread.currentThread().join();

try running sleep in a while loop like:
while(true) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}

running a thread then tell it to sleep in an unstoppable loop, is a good idea,.
but how if you are trying to make it waiting another thread,.? make more than one thread and make them wait one each other, a deadlock condition, is that a hung to,.?

I know what you need exactly, you are testing something through stopping the executor thread. Try something like this:
private void testKillingThread() {
Object kill = new Object();
try {
synchronized (kill) {
kill.wait();
}
} catch (Exception e) {
// Auto-generated catch block
}
}

Simply enough, just create a private member
private Object lock = new Object();
then use it to wait for a notification (that will never happen, unless you use reflection...)
while (true) {
try {
synchronized (lock) {
lock.wait();
}
} cath (InterruptedException e) {
/* ignore interruption */
}
}
and you thread will hang there, uninterruptable.

Here's a quick fix I'm using for testing. Just have the thread you want to lock up call new Hanger().hang().
Remove the logging if you're not interested in seeing it. You can add throws InterruptedException (although, in fact, it never does) to the hang method so you can just replace a Thread.sleep() with a new Hanger().hang() without otherwise modifying your code.
public class Hanger {
private final static Logger log = Logger.getLogger(Hanger.class);
private long started = 0;
private final int beat = 100; // ms
/**
* Hangs a thread for the indicated time
* #param millis the amount of time to hang the thread, in milliseconds
*/
public void hang(int millis) {
started = System.currentTimeMillis();
log.debug("Hanging this thread for " + millis + " ms");
while (hung() < millis) {
try {
Thread.sleep(beat);
} catch (InterruptedException e) {
log.debug("Still hanging, will release in " + (millis - hung()) + " ms.");
}
}
log.debug("Releasing thread again after " + hung() + " ms");
}
private int hung() {
return (int)(System.currentTimeMillis() - started);
}
}