I'm using Spring and I've serveral #Scheduled classes in my application:
#Component
public class CheckHealthTask {
#Scheduled(fixedDelay = 10_000)
public void checkHealth() {
//stuff inside
}
}
#Component
public class ReconnectTask {
#Scheduled(fixedDelay = 1200_000)
public void run() {
//stuff here
}
}
I want the first task use a pool of 2 threads, while the second use a single thread. I don't want the second task is stuck because the first one use all threads available and the computation is slower than fixedDelay time.
Of course mine is just an example to get you the idea.
I could use a configuration class like this:
#Configuration
#EnableScheduling
public class TaskConfig implements SchedulingConfigurer {
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
taskRegistrar.setScheduler(taskScheduler());
}
#Bean
public Executor taskScheduler() {
ThreadPoolTaskScheduler t = new ThreadPoolTaskScheduler();
t.setPoolSize(2);
t.setThreadNamePrefix("taskScheduler - ");
t.initialize();
return t;
}
}
I don't understand how define a different configuration for each #Scheduled component though.
The first task does not require a pool of 2 threads.
Different tasks do not need to be assigned different pools if all using fixed delays. The fixedDelay works as follows:
#Scheduled(fixedDelay=5000)
public void doSomething() {
// something that should execute periodically
}
Would be invoked every 5 seconds with a fixed delay, meaning that the period will be measured from the completion time of each preceding invocation.
Each task only uses one thread, if you have two threads, one thread will not hold up the other to be useable for the other task.
Related
I have a Java application that runs several scheduler to get and provide data to an external application. I will have to add another scheduler to get data from another external application. That would be the exact same process as one of the scheduler already existing for the first application.
So roughly it would be something like this:
However I have small confidence in the formatting of the data of this second application, I know that they have less verifications that the fist application and I might get funny things. I will obviously put plenty of null/bad format check on my side, but I have to make sure that if they ever send me bad data this doesn't impact my others schedulers.
#EnableScheduling
public class myApp{
#Scheduled(fixedRate = 1000)
public void externalApp1() {
do stuff...
commonMethod();
}
#Scheduled(fixedRate = 1000)
public void externalApp2() {
do stuff...
commonMethod();
}
public void commonMethod(){
doStuff...
}
}
One of my first idea is to put dedicated threads to each scheduler, so that if they send bad data and it ends up killing the thread for whatever reason, it only impacts their own process and not the schedulers for the first external application. I have done this for now based on what I found, I suppose this should work as intended:
#Configuration
#EnableAsync
#EnableScheduling
public class MyApp{
#Scheduled(fixedRate = 1000)
#Async(value = "threadPool1")
public void externalApp1() {
dostuff...
commonMethod();
}
#Scheduled(fixedRate = 1000)
#Async(value = "threadPool2")
public void externalApp2() {
dostuff...
commonMethod();
}
public void commonMethod(){
doStuff...
}
#Bean
public Executor threadPool1() {
return Executors.newFixedThreadPool(1);
}
#Bean
public Executor threadPool2() {
return Executors.newFixedThreadPool(1);
}
}
(actual code would be with beans properly separated from main class)
But I am wondering if there is any other way to fully ensure the processes are totally independant from one another?
EDIT: I precise that the data I get from the second application are not used for any process of the first application. It has a process on its own and data are not shared between those 2 external applications
Since I am not working specifically on multi threads, the questions can be low level or even silly, please excuse me =)
Here is my code call flow like;
MessageNotificationJobExecutionConfig -> AsyncMessageNotificationJobExecutor -> NotificationJobExecutor.execute()
MessageNotificationJobExecutionConfig (finds the objects to process) and calls AsyncMessageNotificationJobExecutor inside the loop
AsyncMessageNotificationJobExecutor has #Async("messageNotificationTaskExecutor") annotation over the execute() method.
AsyncMessageNotificationJobExecutor.execute() method calls NotificationJobExecutor.execute()
messageNotificationTaskExecutor is an instance of ThreadPoolTaskExecutor
Here is my question;
If am not wrong as default NotificationJobExecutor has a singletone instance.
Even if AsyncMessageNotificationJobExecutor work async and use thread pool task executor, all thread call only NotificationJobExecutor instance (singletone).
I am not sure, I may misunderstand that Thread_1 calls NotificationJobExecutor.execute() and until this thread finish its job other thread wait for Thread_1. Is my inference correct ?
I think even if it looks multi thread actually it works singletone
#Component("messageNotificationTaskExecutor")
public class MessageNotificationThreadPoolTaskExecutor extends ThreadPoolTaskExecutor {
#Value("${message.notification.task.executor.corePoolSize}")
Integer corePoolSize;
#Value("${message.notification.task.executor.maxPoolSize}")
Integer maxPoolSize;
#Value("${message.notification.task.executor.queueCapacity}")
Integer queueCapacity;
public MessageNotificationThreadPoolTaskExecutor() {
super();
}
#PostConstruct
public void init() {
super.setCorePoolSize(corePoolSize);
super.setMaxPoolSize(maxPoolSize);
super.setQueueCapacity(queueCapacity);
}
}
#Configuration
public class MessageNotificationJobExecutionConfig {
protected Logger log = LoggerFactory.getLogger(getClass());
#Autowired
AsyncMessageNotificationJobExecutor asyncMessageNotificationJobExecutor;
#Autowired
MessageNotificationThreadPoolTaskExecutor threadPoolTaskExecutor;
#Autowired
JobExecutionRouter jobExecutionRouter;
#Autowired
NotificationJobService notificationJobService;
private Integer operationType = OperationType.ACCOUNT_NOTIFICATION.getValue();
#Scheduled(cron = "${message.notification.scheduler.cronexpression}")
public void executePendingJobs() {
List<NotificationJob> nextNotificationJobList = notificationJobService.findNextJobForExecution(operationType, 10);
for (NotificationJob nextNotificationJob : nextNotificationJobList) {
if (threadPoolTaskExecutor.getActiveCount() < threadPoolTaskExecutor.getMaxPoolSize()) {
asyncMessageNotificationJobExecutor.execute(nextNotificationJob);
}
}
}
}
#Service
public class AsyncMessageNotificationJobExecutor {
#Autowired
NotificationJobExecutor notificationJobExecutor;
#Autowired
NotificationJobService notificationJobService;
#Async("messageNotificationTaskExecutor")
public void execute(NotificationJob notificationJob) {
notificationJobExecutor.execute(notificationJob);
}
}
#Component
public class NotificationJobExecutor implements JobExecutor {
#Override
public Integer getOperationType() {
return OperationType.ACCOUNT_NOTIFICATION.getValue();
}
#Override
public String getOperationTypeAsString() {
return OperationType.ACCOUNT_NOTIFICATION.name();
}
#Override
public void execute(NotificationJob notificationJob) {
// TODO: 20.08.2020 will be execute
}
}
In the scenario you created you have all singleton instances. But the flow looks something like this:
call to executePendingJobs in MessageNotificationJobExecutionConfig
iterate over each NotificationJob sequentially (so this is waiting)
call to execute in AsyncMessageNotificationJobExecutor which will add a execution to the messageNotificationTaskExecutor sequential (thus blocking) to the thread pool
execute the job created in step 3 in a separate thread (so this actually executes your method in AsyncMessageNotificationJobExecutor
a blocking call to the execute method in NotificationJobExecutor
The 'magic' happens in step 3, where rather then executing the method Spring will add a job to the messageNotificationTaskExecutor which wraps the call to step 4. This causes the call for step 4 to happen asynchronous and thus multiple calls to the same instance can occur at the same time. So make sure this object is stateless.
My frontend is timing out (504 error) when calling my backend service. This is because my backend service takes ~6 minutes to finish running. I want to return a response to the front-end within a minute and have my backend code continue running after the response is given to the frontend.
I want to use concurrency to run two code segments. One thread will return a String to the frontend, the other thread will call the code that takes around 5 minutes to run on my server.
I want my solution to be simple as this seems like a simple problem to fix, so I am using the simple Executor class from java.util.concurrent
I made my Invoker class as followed:
public class Invoker implements Executor {
#Override
public void execute(Runnable r) {
r.run();
}
}
In my actual code, I have
import java.util.concurrent.Executor;
import com.dcc.standalone.Invoker;
public String aCoolFunction() {
String status = "good job, homie";
Executor executor = new Invoker();
executor.execute( () -> {
// Call this part of the code that takes 5 minutes to run CODE_A
});
return status;
}
I expect status to be returned at the same time CODE_A starts running. Instead, the code runs sequentially as before, i.e., status is returned after CODE_A runs.
maybe use a CompletableFuture?
Setup a ThreadPoolTaskExecutor.
#Configuration
#EnableAsync
public class SpringAsyncConfig {
#Bean(name = "threadPoolTaskExecutor")
public Executor threadPoolTaskExecutor() {
return new ThreadPoolTaskExecutor();
}
}
Define your function
public String aCoolFunction() {
String status = "good job, homie";
someAsyncFunction();
return status;
}
Define a async long running function.
#Async
public void someAsyncFcuntion() {
// Call this part of the code that takes 5 minutes to run CODE_A
}
run your cool function somewhere in a CompletableFuture
String result CompletableFuture.supplyAsync(() -> aCoolFunction()).get();
I'm writing from mobile, but this is what i could come up with from the top of my head.
Declare #Async or create a threadpool using Executors for the service field you want to use. ex)
#Service
public class SimpleService {
private ExectorService pool = Executors.newFixedThreadPool(10);
public String someThing() {
String status = "good job, homie";
pool.execute(() -> {
// other logic
})
return status;
}
}
Suppose there is a button.
Clicking the button disables it (mainThread thread), starts a background task to load data (IO thread). Once data is loaded, the button is enabled back (mainThread thread).
For test, it's common to change schedulers to immediate, but this won't work in my case - button click will block until completion of background task, I'll never be able to check if the button was disabled after it started background task.
Besides unit tests, I'd also like to test this in functional Espresso tests.
How do I test this multithreaded RxJava case?
You can write your own ThreadFactory
ThreadFactory custom = new CustomThreadFactory();
ExecutorService executorService = Executors.newCachedThreadPool(custom); //or use newSingleThreadExecutor(..)
Scheduler customScheduler = Schedulers.from(executorService);
now you can use this scheduler and not block the main queue plus getting called when a new thread is needed:
class CustomThreadFactory implements ThreadFactory {
public Thread lastT;
public int newThreadCounter = 0;
#Override
public Thread newThread(Runnable r) {
newThreadCounter++;
System.out.println("newThread called");
Thread lastT = new Thread(r); //or CustomThread(r)
return lastT;
}
}
You can even go further and instrument the new Thread -
class CustomThread extends Thread {
public CustomThread(Runnable r) {
super(r);
}
#Override
public void run() {
System.out.printf("About to run!");
super.run();
}
}
}
I suggest you to use RxUtil
1). Provide default implementation of RxUtil through the constructor or DI
2). When you create your observable, instead of applying schedulers directly:
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread())
Use RxUtil:
.compose(rxUtil.applySchedulers())
Example:
https://github.com/DAYTeam/GoEnglish/blob/master/app/src/main/java/ru/goenglish/goenglish/services/impl/ScheduleServiceImpl.java#L38-L44
3). In unit tests, instead of default implementation of RxUtil, create mocked version:
public class UnitTestRxUtil implements RxUtil {
#Override
public <T> Observable.Transformer<T, T> applySchedulers() {
return observable -> observable.subscribeOn(Schedulers.immediate())
.observeOn(Schedulers.immediate());
}
}
link: https://github.com/DAYTeam/GoEnglish/blob/master/app/src/test/java/ru/goenglish/goenglish/util/UnitTestRxUtil.java
4). Pass this implementation through the constructor, or DI.
Example (constructor): https://github.com/DAYTeam/GoEnglish/blob/master/app/src/test/java/ru/goenglish/goenglish/ScheduleServiceImplTest.java#L45
As a result, all the tests, will be executed in one thread, and in the application, it will be executed on different executors
I want to write a back-ground job (EJB 3.1), which executes every minute. For this I use the following annotation:
#Schedule(minute = "*/1", hour = "*")
which is working fine.
However, sometimes the job may take more than one minute. In this case, the timer is still fired, causing threading-issues.
Is it somehow possible, to terminate the scheduler if the current execution is not completed?
If only 1 timer may ever be active at the same time, there are a couple of solutions.
First of all the #Timer should probably be present on an #Singleton. In a Singleton methods are by default write-locked, so the container will automatically be locked-out when trying to invoke the timer method while there's still activity in it.
The following is basically enough:
#Singleton
public class TimerBean {
#Schedule(second= "*/5", minute = "*", hour = "*", persistent = false)
public void atSchedule() throws InterruptedException {
System.out.println("Called");
Thread.sleep(10000);
}
}
atSchedule is write-locked by default and there can only ever be one thread active in it, including calls initiated by the container.
Upon being locked-out, the container may retry the timer though, so to prevent this you'd use a read lock instead and delegate to a second bean (the second bean is needed because EJB 3.1 does not allow upgrading a read lock to a write lock).
The timer bean:
#Singleton
public class TimerBean {
#EJB
private WorkerBean workerBean;
#Lock(READ)
#Schedule(second = "*/5", minute = "*", hour = "*", persistent = false)
public void atSchedule() {
try {
workerBean.doTimerWork();
} catch (Exception e) {
System.out.println("Timer still busy");
}
}
}
The worker bean:
#Singleton
public class WorkerBean {
#AccessTimeout(0)
public void doTimerWork() throws InterruptedException {
System.out.println("Timer work started");
Thread.sleep(12000);
System.out.println("Timer work done");
}
}
This will likely still print a noisy exception in the log, so a more verbose but more silently solution is to use an explicit boolean:
The timer bean:
#Singleton
public class TimerBean {
#EJB
private WorkerBean workerBean;
#Lock(READ)
#Schedule(second = "*/5", minute = "*", hour = "*", persistent = false)
public void atSchedule() {
workerBean.doTimerWork();
}
}
The worker bean:
#Singleton
public class WorkerBean {
private AtomicBoolean busy = new AtomicBoolean(false);
#Lock(READ)
public void doTimerWork() throws InterruptedException {
if (!busy.compareAndSet(false, true)) {
return;
}
try {
System.out.println("Timer work started");
Thread.sleep(12000);
System.out.println("Timer work done");
} finally {
busy.set(false);
}
}
}
There are some more variations possible, e.g. you could delegate the busy check to an interceptor, or inject a singleton that only contains the boolean into the timer bean, and check that boolean there, etc.
I ran into the same problem but solved it slightly differently.
#Singleton
public class DoStuffTask {
#Resource
private TimerService timerSvc;
#Timeout
public void doStuff(Timer t) {
try {
doActualStuff(t);
} catch (Exception e) {
LOG.warn("Error running task", e);
}
scheduleStuff();
}
private void doActualStuff(Timer t) {
LOG.info("Doing Stuff " + t.getInfo());
}
#PostConstruct
public void initialise() {
scheduleStuff();
}
private void scheduleStuff() {
timerSvc.createSingleActionTimer(1000l, new TimerConfig());
}
public void stop() {
for(Timer timer : timerSvc.getTimers()) {
timer.cancel();
}
}
}
This works by setting up a task to execute in the future (in this case, in one second). At the end of the task, it schedules the task again.
EDIT: Updated to refactor the "stuff" into another method so that we can guard for exceptions so that the rescheduling of the timer always happens
Since Java EE 7 it is possible to use an "EE-aware" ManagedScheduledExecutorService, i.e. in WildFly:
In for example a #Singleton #Startup #LocalBean, inject the default "managed-scheduled-executor-service" configured in standalone.xml:
#Resource
private ManagedScheduledExecutorService scheduledExecutorService;
Schedule some task in #PostConstruct to be executed i.e. every second with fixed delay:
scheduledExecutorService.scheduleWithFixedDelay(this::someMethod, 1, 1, TimeUnit.SECONDS);
scheduleWithFixedDelay:
Creates and executes a periodic action that becomes enabled first
after the given initial delay, and subsequently with the given delay
between the termination of one execution and the commencement of the
next.[...]
Do not shutdown the scheduler in i.e. #PreDestroy:
Managed Scheduled Executor Service instances are managed by the
application server, thus Java EE applications are forbidden to invoke
any lifecycle related method.
well I had a similar problem. There was a job that was supposed to run every 30 minutes and sometimes the job was taking more than 30 minutes to complete in this case another instance of job was starting while previous one was not yet finished.
I solved it by having a static boolean variable which my job would set to true whenever it started run and then set it back to false whenever it finished. Since its a static variable all instances will see the same copy at all times. You could even synchronize the block when u set and unset the static variable.
class myjob{
private static boolean isRunning=false;
public executeJob(){
if (isRunning)
return;
isRunning=true;
//execute job
isRunning=false;
}
}