I need a mechanism for implementing asynchronous job scheduling in Java and was looking at the Quartz Scheduler, but it seems that it does not offer the necessary functionality.
More specifically, my application, which runs across different nodes, has a web-based UI, via which users schedule several different jobs. When a job is completed (sometime in the future), it should report back to the UI, so that the user is informed of its status. Until then, the user should have the option of editing or cancelling a scheduled job.
An implementation approach would be to have a Scheduler thread running constantly in the background in one of the nodes and collecting JobDetail definitions for job execution.
In any case, there are two the questions (applicable for either a single-node or multi-node scenario):
Does Quartz allow a modification or a cancellation of an already scheduled job?
How can a "callback" mechanism be implemented so that a job execution result is reported back to the UI?
Any code examples, or pointers, are greatly appreciated.
Does Quartz allow a modification or a cancellation of an already scheduled job?
You can "unschedule" a job:
scheduler.unscheduleJob(triggerKey("trigger1", "group1"));
Or delete a job:
scheduler.deleteJob(jobKey("job1", "group1"));
As described in the docs:
http://quartz-scheduler.org/documentation/quartz-2.x/cookbook/UnscheduleJob
Note, the main difference between the two is unscheduleing a job removes the given trigger, while deleting the job removes all triggers to the given job.
How can a "callback" mechanism be implemented so that a job execution result is reported back to the UI?
Typically, in the web world, you will poll the web server for changes. Depending upon your web framework, there may be a component available (Push, Pull, Poll?) that makes this easy. Also, what you will need to do is store some state about your job on the server. Once the job finishes you may update a value in the database or possibly in memory. This, in turn, would be picked up by the polling and displayed to the user.
To complete the answer of #johncarl with Quarz also have this
handle event's before and after the excution implementing SchedulerListener interface
Groups your jobs
Support with Enterprise Edition
Most important the cron expressions here's the documentation
Related
Our Spring Web Application uses Spring Batch with Quartz to carry out complex jobs. Most of these jobs run in the scope of a transaction because if one part of the complex system fails we want any previous database works to be rolled back. We would then investigate the problem, deploy a fix, and restart the servers.
It's getting to be an issue because some of these jobs do a HUGE amount of processing and can take a long time to run. As execution time starts to surpass the 1 hour mark, we find ourselves unable to deploy fixes to production for other problems because we don't want to interrupt a vital job.
I have been reading up on the Reactor implementation as a solution to our problems. We can do a small bit of processing, publish an event, and have other systems do the appropriate action as needed. Sweet!
The only question I have is, what is the best way to handle failure? If I publish an event and a Consumer fails to conduct some critical functionality, will it restart at a later time?
What if an event is published, and before all the appropriate consumers that listen for it can handle it appropriately, the server shuts down for a deployment?
I just started to use reactor recently so I may have some misconception about it, however I'll try to answer you.
Reactor is a library which helps you to develop non-blocking code with back-pressure support which may help you to scale your application without consuming a lot of resources.
The fluent style of reactor can easily replace Spring Batch however the reactor by itself doesn't provide any way to handle transaction nor Spring and in case the jdbc current implementation it will be always blocking since there's no support in the drive level to non-blocking processing. There are discussions around how to handle transactions anyway but as far as know there's no final decision about this matter.
You can always use transactions but remember that you are not going to have non-blocking processing since you need to update/delete/insert/commit in the same thread or manually propagate the transactional context to the new thread and block the main thread
So I believe Reactor won't help you solve your performance issues and another kind of approach may take place.
My recommendation is:
- Use parallel processing in Spring Batch
- Find the optimal chunk number
- Review your indexes (not just create but delete it)
- Review your queries
- Avoid unneeded transformations
- And even more important: Profile it! the bottleneck can be something that you have no idea
Spring batch allows you to break large transactions up into multiple smaller transactions when you use chunk oriented processing. If a chunk fails, then it's transaction rolls back but all previous chunks transactions would have committed. By default, when you restart the job, it will start again from where it failed, so if it had already processed 99 chunks successfully and the 100th chunk failed, restarting the job starts from the 100th chunk and continues.
If you have a long running job and want to deploy a new version, you can stop the job and it will stop after processing the current chunk. You can then restart the job from where it was stopped. It helps to have a GUI to view, launch, stop and restart your jobs. You can use spring batch admin or spring cloud dataflow for that if you want an out of the box GUI.
I am creating a web application and in this I will be creating many services and executors to do some tasks.I have extended dispacter servlet and started executors or other threads in init method.Is this the right approach?
Now Suppose if any request comes and that executor or similar task executing thread dies after throwing Exception.
1.I suppose that it will affect other requests also.So what shall I do in such cases?
2.How can I create a monitor thread which will check if all critial tasks executing thread and executors are properly running?
3.Should I keep another backup executor prepared and deferred to takeover the failed executor in such situations?If so then how?
It is an old one, but maybe it would help someone :)
For ExecutorService there is a nice example on how to approach the problem in Codahale Metrics: https://github.com/dropwizard/metrics/blob/master/metrics-core/src/main/java/com/codahale/metrics/InstrumentedExecutorService.java
I did not find anything as good for the Spring AsyncTaskExecutors :/
Its been a while since I have used an Executor, are you using one of the built-in executors from Java, Spring, or are you rolling your own? Spring has a bunch, and I think Java gives you two or three concrete implementations.
Anyhow, I think the answer would be to roll some sort of monitoring service, like maybe something using JMX if you have that available. If you wanted to wire your Executors auto-magically you could use the ApplicationContextAware interface to get a reference to the ApplicationContext, which has a method called getBeansOfType(). If you want to take the more straight-forward approach, then you can simply write your monitoring service to inject the executors in there directly.
Another option would be to get an external monitoring framework/app. Something like Dynatrace, which attaches to the JVM process and monitors things or, if you don't mind switching app servers, SpringSource's tcServer, which has optional instrumented Spring JARs and provides a ton of out-of-the-box monitoring.
I'm trying to make a mini web application for reminders, I deploy Quartz Scheduler to handle the issue of launch events reminder, I have understood the tasks (Jobs) and programmers (Schedulers) can be configured from a Database with JDBC, I have searched and can not find an example where I show what information should I put on the tables and I run java code to start operating scheduled tasks. If someone can have an example or something that I can serve this purpose, they are grateful.
You have understood wrong. You can use any JobStore (including the JdbcJobStore to store your jobs/triggers/etc. but creating them manually in the database is a bad idea™.
Depending on how you are using Quartz you can set it up, either using SPRING or using the Fluent syntax (which I believe is the preferred method these days).
Further reading: http://quartz-scheduler.org/documentation/quartz-2.1.x/tutorials/tutorial-lesson-09
I have a quite an update-intensive application and was wondering if it is a good idea to have a daemon thread that has a hibernate session and that takes update instructions and groups those updates into a batch (which I guess is more efficient).
Thank you in advance.
If your use-cases trigger many updates, but those updates are not necessary to be executed right when they are triggered (like the data they would add is not mandatory on the spot) you can and in fact it is advised to store them and execute them at a later time (like at night, or when the application is less heavily hit). You don't necessarely need a daemon (or any kind of) thread for that, just store your update requests somewhere, and then have a CRON Job or something be triggered at certain intervals and execute them. You can use Quartz Scheduler for this, it will deal with all the underlying threads/scheduling.
Also regarding batching multiple queries vs. executing them idividually, normally Hibernate takes care of that (like if you issue multiple update requests in the same transaction Hibernate will batch them togehter and execute them at the latest possible time - usually when the transactio ends), so this shouldn't be a concern for you. What MIGHT become a concern, depening on your number of update requests , would be Hibernate itself. At some point you might want to consider using JDBC and batch execute those updates manually to gain performance.
I believe any programmer who has been dealing with database requests in a gui application has run into some or all of the following problems:
Your GUI freezes because you call
database layer from within the event
dispatch thread
When you have multiple windows/panels/jframes where user can start a db request your performance degrades because you don't have any control about the threads your user creates
User may be able to lock down the application and even the database because he calls any action many times before the first action has been finished
What I'd like to know about: are there any frameworks that handle the requirements of handling an ordered set of long running actions (including but not limited to database calls, i.e. calculations) outside the event dispatch thread?
Note: I know of SwingWorker ;-)
Naked Objects facilitate a clean domain model and they also have a GUI 2 DB mapping layer -- http://www.nakedobjects.org/home/index.shtml
Such a thing should be found in Netbeans for example. See RequestProcessor. But in simpler cases this is not required. Last time I need something like thread scheduling and control I simply used new concurrency packages included in J5 (I used J6). With its ExecutorFactory-ies you can simply achieve basic control over tasks. You can also use some queues. This PDF can help. The PDF is written in Slovak language but the Single/Multiple task workers are there written in Java ;)
I doubt you will find something specific for database requests. You can try to reuse existing generic task scheduling libraries. An example is the Eclipse jobs API. This does not depend on the IDE.
See http://www.eclipse.org/articles/Article-Concurrency/jobs-api.html