we need run one function periodically in Java web application .
How to call function of some class periodically ?
Is there any way that call function when some event occured like high load in server and so on . what is crontab ? Is that work periodically ?
To call something periodically, see TimerTask
If you need something more robust you can use Quartz
As for crontab is the scheduling tool on Unix machines.
For calling methods when server has high load, you have at least two possible approaches. Your App Server might have management hooks that would you allow to monitor its behaviour and take progrommatic action. Alterntaively you have some system monitoring capability (Eg. Tivoli or OpenView) and it generates "events", it should not be too hard to deliver such events as (for example) JMS messages and have your server pick them up.
However, you might want to explain a bit more about what you want to achieve. Adaptive application beahviour might be quite tricky to get right.
If you want to run something periodically, don't do it in the webserver. That would be a very wrong approach IMO. It's better to use cron instead, if you are on a Unix-like operating system. Windows servers offer similar functionality.
we need run one function periodically
in Java web application
(1) So look in your deployment descriptor (web.xml) define a listener to startup at startup time.
How to call function of some class
periodically ?
(2) Create a Timer in the listener.
Is there any way that call function
when some event occured like high load
in server and so on
(3) and run some Threads to check for system conditions that are accesible with Java, even run system progs (uptime, etc) and parse the output.
crontab could be a way but the execution of Java will start another JVM and that is really the hot thing in servlet containers: all runs in the same JVM.
Don't forget about java.util.concurrent - it has a lot of classes for scheduling, e.g. ScheduledThreadPoolExecutor, if you need more than a simple Timer.
http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/package-summary.html
There is also a backport of it to Java 1.4: http://backport-jsr166.sourceforge.net.
If you already use Spring, you might want to have a look at Spring's task execution framework - using #Scheduled and #Async for annotating methods as tasks and implementing the functionality in a Processor that delegates the actual work to a Worker, as described in:
http://blog.springsource.com/2010/01/05/task-scheduling-simplifications-in-spring-3-0/
The advantage is that you can define timers using a cron-like syntax in your spring context and you don't need anything special to set up tasks, it is also well integrated into Java EE applications and should play well with web servers (which custom Threads tend not to do always).
How to call function of some class periodically?
There are several solutions: a Timer, a Java cron implementation like cron4j, Quartz, or even the EJB Timer API. Choosing one or the other highly depends on the context: the type of application, the technologies used, the number of jobs, etc.
Is there any way that call function when some event occurred like high load in server and so on
You could maybe use JMX in your jobs to access and monitor informations and trigger an action under some specific condition. But this is more a pull mode, not event based.
Related
We have a Java application deployed in open shift container platform. There are 4 identical stateless pods running the application. In our application , a scheduler runs every 15 mins that process some data. I need to perform the scheduler run only in one of the pod. As doing it in all the pods leads to duplicate operations. How can I achieve this? I am looking for some best and simplest way to achieve this without complicating much. Please suggest me some options.
Does it really need to run in one of the pods? Because as you are discovering, that's somewhat problematic. By definition, the pods are identical and so to go down that path you you are going to have to use some kind of external coordinator like a database or PV.*
On the other hand if you use something like a CronJob, you can let OpenShift do the work for you regarding both the scheduling and the "only happen once". You'll also get better visibility, logging, and troubleshooting. You can probably even re-use your image, perhaps with just a different CMD.
*Technically this isn't true, you could use some kind of consensus mechanism within your pods, but that's a lot of complexity if you just want a cron job.
Use distributed locks.If the lock fails, other pods are executing tasks.
I'm writing an java based app (not web app) and it should be able to run standalone without any container the task it carries are below:
windows scheduler fires off either quartz or simple POKO
pick up file(s) during midnight
import the data into DB
move the files over from original destination to another drive
Now, the dilemma I'm having is I've been reading around and it appears quartz need web container to function.
Is that correct AND what would be most simple and durable solution?
According your question: Quartz does not need a web container, it can be run in any java application. See Quartz Quickstart Guide for how to configure Quartz.
If you use Quartz the windows scheduler shouldn't be necessary, but this implies that your java application is running constantly.
I think Quartz has the advantage, that you can configure your application in one place and do not need to consider os specific scheduling. Further more Quartz makes you independent of the os specific scheduling mechanism.
But: All this advantages are not relevant if your application is not running all the time.
On the other hand if you want it to be a fire and forget like application, that runs, does its work and then quits again, you will be on the safe side to delegate the task of scheduling to the operation system your application runs on.
So, for this specific context I think using the operation system's scheduling mechanism is the better option.
In a java web application (servlets/spring mvc), using tomcat, is it possible to run a cron job type service?
e.g. every 15 minutes, purge the log database.
Can you do this in a way that is container independent, or it has to be run using tomcat or some other container?
Please specify if the method is guaranteed to run at a specific time or one that runs every 15 minutes, but may be reset etc. if the application recycles (that's how it is in .net if you use timers)
As documented in Chapter 23. Scheduling and Thread Pooling, Spring has scheduling support through integration classes for the Timer and the Quartz Scheduler (http://www.quartz-scheduler.org/). For simple needs, I'd recommend to go with the JDK Timer.
Note that Java schedulers are usually used to trigger Java business oriented jobs. For sysadmin tasks (like the example you gave), you should really prefer cron and traditional admin tools (bash, etc).
If you're using Spring, you can use the built-in Quartz or Timer hooks. See http://static.springsource.org/spring/docs/2.5.x/reference/scheduling.html
It will be container-specific. You can do it in Java with Quartz or just using Java's scheduling concurrent utils (ScheduledExecutorService) or as an OS-level cron job.
Every 15 minutes seems extreme. Generally I'd also advise you only to truncate/delete log files that are no longer being written to (and they're generally rolled over overnight).
Jobs are batch oriented. Either by manual trigger or cron-style (as you seem to want).
Still I don't get your relation between webapp and cron-style job? The only webapp use-case I could think of is, that you want to have a HTTP endpoint to trigger a job (but this opposes your statement about being 'cron-style').
Generally use a dedicated framework, which solves the problem-area 'batch-jobs'. I can recommend quartz.
The application has a CPU intensive long process that currently runs on one server (an EJB method) serially when the client requests it.
It’s theoretically possible (from a conceptual point of view) to split that process in N chunks and execute them in parallel, as long as the output of all parallel jobs can be collected and joined together before sending it back to the client that initiated the process. I’d like to use this parallelization to optimize performance.
How can I implement this parallelization with EJBs? I know that we should not create threads in a EJB method. Instead, we should publish messages (one per job) to be consumed by message driven beans (MDBs). But then it would not be a synchronous call anymore. And being synchronous seems to be a requirement in this case since I need to collect the output of all jobs before sending it back to the client.
Is there a solution for this?
There are all sorts of ways to do this.
One, you can use an EJB Timer to create a run-once process that will start immediately. This is a good technique to spawn processes in the background. A EJB Timer is associated with a specific Session Bean implementation. You can either add an EJB Timer to every Session Bean that you want to be able to do this, or you can have a single Session Bean that can then call your application logic through some dispatch mechanism.
For me, I pass a serializable blob of parameters along with a class name that meets a specific interface to a generic Session Bean that then executes the class. This way I can easily background most anything.
One caveat about the EJB Timer is that EJB Timers are persistent. Once you create an EJB Timer is stays in the container until its job is finished or canceled. The gotcha on this is that if you have a long running process, and the server goes down, when it restarts the process will continue and pick back up. Mind this can be a good thing, but only if your process is prepared to be restarted. But if your have a simple process iterating through "10,000 items", if the server goes down on item 9,999, when it comes back up you can easily see it simply starting over at item 1. It's all workable, just a caveat to be aware of.
Another way to background something is you can use a JMS queue. Put a message on the queue, and the handler runs aysnchronously from the rest of your application.
The clever part here, and something I has also done leveraging the work with the Timer Bean, is you can control how many "jobs" will run based on how many MDB instances you configure the system to have.
So, for the specific task of running a process in multiple, parallel chunks, I take the task, break it up in to "pieces", and then send each piece on the Message Queue, where the MDBs execute them. If I allow 10 instances of the MDB, I can have 10 "parts" of any task running simultaneously.
This actually works surprisingly well. There's a little overhead it splitting the process up and routing it through the JMS queue, but that's all basically "start up time". Once it gets going, you get a real benefit.
Another benefit of using the Message Queue is you can have your actual long running processes executing on a separate machine, or you can readily create a cluster of machines to handle these processes. Yet, the interface is the same, and the code doesn't know the difference.
I've found once you've relegated a long running process to the background, you can pay the price of having less-that-instant access to that process. That is, there's no reason to monitor the executing classes themselves directly, just have them publish interesting information and statistic to the database, or JMX, or whatever rather than having something that can monitor the object directly because it shares the same memory space.
I was easily able to set up a framework that lets task run either on the EJB Timer or on the MDB scatter queue, the tasks are the same, and I could monitor their progress, stop them, etc.
You could combine the scatter technique to create several EJB Timer jobs. One of the free advantages of the MDB is it acts as a thread pool which can throttle your jobs (so you don't suddenly saturate your system with too many background processes). You get this "for free" just by leveraging the EJB management features in the container.
Finally, Java EE 6 has a new "asynchronous" (or something) qualifier for Session Bean methods. I do not know the details on how this works, as I've yet to play with a new Java EE 6 container. But I imagine you're probably not going to want to change containers just for this facility.
This particular question has come up on multiple occasions and I will summarize that there are several possible solutions, only 1 of which I would recommend.
Use a WorkManager from the commonj API. It allows for managed threads in a Java EE container and is specifically designed to fit your use case. If you are using WebSphere or WebLogic, these API's are already available in your server. For others your will have to put a third party solution in yourself.
WorkManager info
Related questions
Why Spawning threads is discouraged
An EJB is a ultimately a transactional component for a client-server system providing request/reply semantics. If you find yourself in the position that you need to pigeonhole a long-running transaction within the bounds of a request/reply cycle, then somewhere your system architect(ure) has taken the wrong turn.
The situation you describe is cleanly and correctly handled by an event based architecture with a messaging back end. Initial event initiates the process (which can then be trivially parallelized by having the workers subscribe to the event topic) and the aggregating process itself raises an event on its completion. You can still squeeze these sequence within the bounds of a request/reply cycle, but you will by necessity violate the letter and spirit of the Java EE system architecture specs.
Back to the Future - Java EE 7 has lot more Concurrency support via ManagedThreadFactory, ManagedExecutor service etc (JSR 236: Concurrency Utilities for Java EE) with which you can create your own 'managed'Threads .It is no longer a taboo in EE AS supporting it (Wildfly ?) via usining the ManagedThread* API's
More details
https://jcp.org/aboutJava/communityprocess/ec-public/materials/2013-01-1516/JSR236-EC-F2F-Jan2013.pdf
http://docs.oracle.com/javaee/7/tutorial/doc/concurrency-utilities002.htm
I once participated in a project where EJB transactions ran for up to 5 hours at a time. Aargh!
This same application also had a BEA specialist consultant who approved that they started additional threads from the transactions. While it's disrecommended in the specs and elsewhere, it doesn't automatically result in failure. You need to be aware that your extra threads are outside the container's control and thus if something goes wrong it's your fault. But if you can assure that the number of threads started in the worst case doesn't exceed reasonable limits, and that they all terminate cleanly within reasonable time, then it is quite possible to work like this. In fact, in your case it sounds like the almost-only solution.
There are some slightly esoteric solutions possible where your EJB app reaches out to another app for a service, which then does the multithreading in itself before returning to the EJB caller. But this is essentially just shifting the problem around.
You may, however, consider a thread pooling solution to keep an upper limit on the number of threads spawned. If you have too many threads your application will behave horribly.
You've analyzed the situation quite well, and no, there is not patern for this that match the EJB model.
Creating threads is mainly forbidden because it bypass the app. server thread management strategy and also because of the transactions.
I worked on a project with similar requireements and I decided to spawn additional threads (going against the sepc then). The operation to parallelized was read-only, so it worked regarding the transaction (the thread would basically have not transaction associated to them). I also knew that I wouldn't spawn too many threads per EJB calls, so the number of threads was not an issue. But if your threads are supposed to modify data, then you break the transactional model of the EJB seriously. But if your operation in pure computing, that might be ok.
Hope it helps...
This app must perform connection to a web service, grab data, save it in the database.
Every hour 24/7.
What's the most effective way to create such an app in java?
How should it be run - as a system application or as a web application?
Keep it simple: use cron (or task scheduler)
If that's all what you want to do, namely to probe some web service once an hour, do it as a console app and run it with cron.
An app that starts and stops every hour
cannot leak resources
cannot hang (may be you lose one cycle)
consumes 0 resources 99% of the time
look at quartz, its a scheduling library in java. they have sample code to get you started.
you'd need that and the JDBC driver to your database of choice.
no web container required - this can be easily done using a stand alone application
Try the ScheduledExecutorService.
Why not use cron to start the Java application every hour? No need to soak up server resources keeping the Java application active if it's not doing anything the rest of the time, just start it when needed,
If you are intent on doing it in java a simple Timer would be more than sufficient.
Create a web page and schedule its execution with one of many online scheduling services. The majority of them are free, very simple to use and very reliable. Some allows you to create schedules of any complexity just like in cron, SqlServer job UI, etc. Saves you a LOT of headache creating/debugging/maintaining your own scheduling engine, even if it's based on some framework like Ncron, Quartz, etc. I'm speaking from my own experience.