I'm using 3rd party library for doing some work in my application. Unfortunately, some bugs were found in it and they cause very sad result: my app is hanging as worker thread probably infinite looping. I've read some questions about killing the thread in android VM but they are didn't help me because of:
stop() method is deprecated and not supported bu Andriod VM
interrupt() method doesn't do anything, i mean thread is still alive
The worst thing is that from some moment of time this worker thread starts to use a lot of memory causing GC to run too often which also is not good for app.
I've found some conditions when bug in library is occurred but i there are may be other bugs which i also want to avoid in my app.
Here is ode snippet that shows the problem:
final MutableObject<Object> calculationResult = new MutableObject<Object>(null);
final MutableObject<EvalError> exception = new MutableObject<EvalError>(null);
final MutableObject<Thread> calculationThread = new MutableObject<Thread>(null);
final CountDownLatch latch = new CountDownLatch(1);
new Thread(new Runnable() {
#Override
public void run() {
final Thread thread = Thread.currentThread();
try {
Log.d(CalculatorEngine.class.getName(), "Calculation thread started work: " + thread.getName());
calculationThread.setObject(thread);
calculationResult.setObject(interpreter.eval(jsclExpression));
} catch (EvalError evalError) {
exception.setObject(evalError);
} finally {
Log.d(CalculatorEngine.class.getName(), "Calculation thread ended work: " + thread.getName());
calculationThread.setObject(null);
latch.countDown();
}
}
}).start();
try {
Log.d(CalculatorEngine.class.getName(), "Main thread is waiting: " + Thread.currentThread().getName());
latch.await(4, TimeUnit.SECONDS);
Log.d(CalculatorEngine.class.getName(), "Main thread got up: " + Thread.currentThread().getName());
final EvalError evalErrorLocal = exception.getObject();
final Object calculationResultLocal = calculationResult.getObject();
final Thread calculationThreadLocal = calculationThread.getObject();
if (calculationThreadLocal != null) {
// todo serso: interrupt doesn't stop the thread but it MUST be killed
calculationThreadLocal.interrupt();
resetInterpreter();
}
if ( evalErrorLocal != null ) {
throw evalErrorLocal;
}
if ( calculationResultLocal == null ) {
tooLongExecutionCache.add(jsclExpression);
throw new ParseException("Too long calculation for: " + jsclExpression);
}
} catch (InterruptedException e) {
throw new ParseException(e);
}
Thread could not be killed in davlik:
I used solution proposed by AVD and just set the priority of thread to lowest possible value and invoke interrupt() method.
System.exit() might at least kill your application, if you can detect that the task has stopped responding.
Related
I am using resilience4j Timelimiter in my project.
The timelimiter is throwing an error if a request is taking more than 10s, but it is not interrupting the thread.
When call comes from postman, i have put the debug and tested, after 10s in postman it displays an exception, but the thread still executes the method and after that added some print statements and it executed as well.
How to cancel or interrupt the thread after 10s in resilience4j.
class A {
TimeLimiterConfig config = TimeLimiterConfig.custom().cancelRunningFuture(true)
.timeoutDuration(Duration.ofMillis(TimeLimit)).build();
TimeLimiterRegistry timeLimiterRegistry = TimeLimiterRegistry.of(config);
TimeLimiter timeLimiter = timeLimiterRegistry.timeLimiter("APITimelimiter", config);
public Response someMethod() throws Exception {
try {
timeLimiter.executeFutureSupplier(() -> CompletableFuture.supplyAsync(() -> {
return getData();
}));
} catch (Exception e) {
logger.error("Request has crossed the execution time of " + TimeLimit
+ " seconds");
throw new Exception("Your request has crossed the execution time of "+ TimeLimit+" seconds.");
}
}
public UserData getData() {
String jsonData = "";
return jsonData;
}
}
TimeLimiter cannot cancel a CompletableFuture. See #TimeLimiter times out slow method but does not cancel running future #905 Points out, that: the limited cancel() in case of CompletableFuture is not a bug, but a design decision. CompletableFuture is not inherently bound to any thread, while Future almost always represents background task.
In a Spring Boot service class, let's say that I am making a method call processEvent().
The method processEvent() might be doing N number of things including making REST calls to other services.
How to check the time being taken by the method in parallel and if it crosses the threshold, then do something else e.g. throw exception ?
class EventService {
public void processEvent(ServiceContext context, Event event) {
// Field a time checker here for the below method.
processEvent(event);
}
public void processEvent(Event event) {
// this method does many things.
}
}
Can this be achieved using CompletionService ? If yes, Please give an example!
EDIT:
The following code works but I have one query:
public void processEvent(ServiceContext context, Event event) {
LOGGER.debug("Timestamp before submitting task = {}", System.currentTimeMillis());
Future<EventResponse> future = executor.submit(() -> {
LOGGER.debug("Timestamp before invoking = {}", System.currentTimeMillis());
EventResponse eventResponse = processEvent(event);
LOGGER.debug("Timestamp after invoking = {}", System.currentTimeMillis());
return eventResponse;
});
try {
LOGGER.debug("Thread sleep starts at = {}", System.currentTimeMillis());
Thread.sleep(5000);
LOGGER.debug("Thread sleep ended at = {}", System.currentTimeMillis());
} catch (InterruptedException e) {
LOGGER.debug("Going to print stack trace....");
e.printStackTrace();
}
if (!future.isDone()) {
future.cancel(true);
LOGGER.debug("task executor cancelled at = {}", System.currentTimeMillis());
} else {
EventResponse response = future.get();
LOGGER.debug("Received Event ID = {}", response.getEventDetailsList().get(0).getEventID());
return response;
}
LOGGER.debug("Going to return error response at = {}", System.currentTimeMillis());
throw new Exception("Message");
}
I am getting the below logs:
Timestamp before submitting task = 1579005638324
Thread sleep starts at = 1579005638326
Timestamp before invoking = 1579005638326
Thread sleep ended at = 1579005638526
task executor cancelled at = 1579005638527
Going to return error response at = 1579005638527
Timestamp after invoking = 1579005645228
How "Timestamp after invoking" is logged after "task executor cancelled at" ?
You can use ThreadPoolTaskExecutor to submit the task, then sleep for a certain amount of time, then check if the task is completed and interrupt it, if it's still working. However, you can't just kill the task, you'll have to periodically check for the interrupted flag inside the task itself. The code would be something like:
#Autowired
private ThreadPoolTaskExecutor executor;
// ...
Future<?> future = executor.submit(() -> {
doOneThing();
if(Thread.interrupted()) {
return;
}
doAnotherThing();
if(Thread.interrupted()) {
return;
}
// etc.
});
Thread.sleep(10000);
if (!future.isDone()) {
future.cancel(true);
}
You can use a mix of a standard ThreadPoolExecutor with a ScheduledThreadPoolExecutor. The latter will cancel the submission of the former if it's still running.
ThreadPoolExecutor executor = ...;
ScheduledThreadPoolExecutor watcher = ...;
Future<?> future = executor.submit(() -> { ... })
watcher.schedule(() -> future.cancel(true), THRESHOLD_SECONDS, TimeUnit.SECONDS);
The future.cancel(true) will be a no-op if it completed. For this though, you should be aware of how to handle cross-thread communiccation and cancellation. cancel(true) says "Either prevent this from running entirely, or, if it is running, interrupt the thread indicating we need to stop execution entirely and immediately"
From there your Runnable should handle interruption as a stop condition:
executor.submit(()-> {
// do something
if(Thread.currentThread().isInterrupted()) {
// clean up and exit early
}
// continue doing something
});
While going through an investigation in a legacy Java Spring Maven project deployed on tomcat 7 the logs stated like below-
2018-08-29 18:16:42:471 +0600 [http-bio-8080-exec-494] [ERROR]
Asking to demystify the number after
exec-
So basically the meaning of "exec"?
which is 494 for the above case.
It's most probably the thread id generated by a custom ThreadFactory, just like:
Executor executor = Executors.newFixedThreadPool(4, new ThreadFactory() {
AtomicInteger threadId = new AtomicInteger(0);
#Override
public Thread newThread(Runnable r) {
return new Thread(r, "http-bio-8080-exec-" + threadId.getAndIncrement()); // custom a thread factory
}
});
IntStream.range(0, 10).forEach(value -> {
executor.execute(() -> {
System.out.println(Thread.currentThread().getName()); // print thread name
try {
Thread.sleep(100);
} catch (Exception e) {
}
});
});
OutPut:
http-bio-8080-exec-0
http-bio-8080-exec-1
http-bio-8080-exec-2
http-bio-8080-exec-3
http-bio-8080-exec-0
http-bio-8080-exec-3
http-bio-8080-exec-1
http-bio-8080-exec-2
http-bio-8080-exec-0
http-bio-8080-exec-3
That is a Thread ID number generated by a Thread Pool in tomcat. The real question is different, this being an internal information what is the value of it, now that you know about it? I'd assume close to zero...
This may be bit old question. I am confused with the ExecutorService work in Jboss environment. I used some sample code, where i am submitting the task with ExecutorService and after everything is done i am shutdown the executor.
Problem i am facing is after submitting one request, i am getting below exception for subsequent request.
Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask#518ad6a2 rejected from java.util.concurrent.ThreadPoolExecutor#72114f80[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 0]
ExecutorService executorService = Executors.newFixedThreadPool(3);
#POST
#Path("/request")
public Response checkAsync(final MultiMedia multiMedia) {
final Random rand = new Random();
final String random = String.valueOf(rand.nextInt(50) + 1);
multiMediaJobs.put(random, multiMedia);
final String jobId = "{ 'jobId' : " + random + "}";
executorService.submit(new Runnable() {
#Override
public void run() {
boolean result = veryExpensiveOperation(jobId);
if (result) {
try {
MultiMedia multiMedia = (MultiMedia) multiMediaJobs.get(random);
multiMedia.getMediadata().getMetadata()
.setAssetId(random);
final String uri = multiMedia.getCallback().getUri()+multiMedia.getCallback().getResource();
RestTemplate restTemplate = new RestTemplate();
String code = restTemplate.postForObject(uri,
multiMedia, String.class);
System.out.println(code);
} finally {
logger.debug("Map size: " + multiMediaJobs.size());
logger.debug("Time: "+System.currentTimeMillis());
multiMediaJobs.remove(random);
}
}
}
private boolean veryExpensiveOperation(String jobId) {
try {
Thread.sleep(7000);
} catch (InterruptedException e) {
e.printStackTrace();
}
logger.debug("Task is processed fully");
return true;
}
});
executorService.shutdown();
try {
executorService.awaitTermination(1, TimeUnit.SECONDS);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return Response.status(Status.ACCEPTED)
.entity(commonHelper.toJSON(jobId)).build();
}
Is it really required to call shutdown in JBOSS environment? If i remove that it is accept all my request. Example i am seeing in all here is just main method. I just want to know how it works in real application.
Forgive me if i am misunderstood some concept.
The problem is that you shutdown the ExecutorService. So any subsequent task being submitted is rejected right away.
I think you have some misunderstanding here.
When you submit to an executor, you will normally get a Future<T> object back. If you need a result from this, you'll call Future.get() and that will block until the threadpool executes your job. Otherwise you can just leave your jobs to be executed.
You wouldn't normally shutdown the executor unless you really want to shut it down, not accept any jobs, and let those queued up execute.
I am trying to stop Tomcat Service and a custom service (which i have created in windows) using JNA in my Java program. Sometimes Tomcat service takes longer time to stop.
W32Service service = null;
long startTime = System.currentTimeMillis();
boolean isStopped = false;
try
{
W32ServiceManager serviceManager = new W32ServiceManager();
serviceManager.open(Winsvc.SC_MANAGER_ALL_ACCESS);
service = serviceManager.openService(serviceName, Winsvc.SC_MANAGER_ALL_ACCESS);
//service.queryStatus().dwWaitHint = 60000;
synchronized (service) {
service.stopService();
}
service.close();
} catch (Exception ex)
{
System.out.println("Timeout happened.");
}finally{
if(null != service.getHandle()){
while(!isStopped){
if(service.queryStatus().dwCurrentState == Winsvc.SERVICE_STOPPED
|| System.currentTimeMillis()-startTime >= Constants.SERVICE_STOP_WAIT_TIME){
isStopped=true;
}
}
if(System.currentTimeMillis()-startTime >= Constants.SERVICE_STOP_WAIT_TIME){
System.out.println("Service Continuing.....");
throw new ServiceNotStoppedException("Service Not Stopped after 2 minutes");
}else{
System.out.println("Service Stopped");
isStopped=true;
}
}else{
System.out.println("Service Stopped.");
isStopped=true;
}
}
In the above code, i am catching a Runtime Exception thrown which give the below message.
java.lang.RuntimeException: Timeout waiting for service to change to a non-pending state.
at com.sun.jna.platform.win32.W32Service.waitForNonPendingState(W32Service.java:162)
at com.sun.jna.platform.win32.W32Service.stopService(W32Service.java:98)
at com.taskkill.sample.Sample.stopService(Sample.java:32)
at com.taskkill.sample.Sample.main(Sample.java:12)
I have checked previous threads on similar topic but couldn't find any resolution. It is waiting for a default time and throwing error. Is there any way we can extend this time dwWaithint? (I have tried //service.queryStatus().dwWaitHint = 60000;) but it doesn't work. Tomcat Service is eventually stopping/Hung while using this method. This works but is there a better way to handle the RunTimeException?
If i use command prompt process by the following code, it returns exit code "0" immediately, whereas the process is still stopping in the background.
String[] stopTomcat = { "sc", "stop", "Tomcat7" };
Process tomcatProcess = Runtime.getRuntime().exec(stopTomcat);