I am building a web console that will be used to monitor the progress of an ongoing tasks of processing for multiple files and my requirement is to have a realtime updates for each and every file separatly (based on user request) via Web-sockets, so the user will typically login and select the file he want to see the progress on and will be re-directed to a page with live updates from the system about this file so typically I will need a different topic to be created every time based on user request with my scheduled job running on the background publishing updates on each created topic.
my question is there any way to fulfill these requirement using spring WebSockets and scheduled jobs or should I switch to event bus like the one in vertx ?
Regards,
After a week of trials i ended up using vertx eventbus, it gives me exactly what i needed, it even includes a periodic timer which can be cancelled by a timer id which is exactly what i was looking for, adding the Async nature of vertx and its lightweight which makes it a perfect fit for my requirements.
Related
I need suggestions regarding creating a scheduled task in a spring boot application. Our application is an order management portal that helps users with details about an order or a list of orders.
Currently, there is a manual process that is as followed: when a new order comes into the database, the user picks up some information of the order, prepares a doc file of it, and sends it to an external portal through email. That portal sends users a response in the email and the user saves the response in the system through our application. Now we want to automate this process in which instead of going the email route we will make a SOAP call to the external portal(send the doc as an attachment in XML request) and get the response back. For this, I want to write a scheduled process (#Scheduled) which will pick up the new orders and make a SOAP call for them.
My question is should I create the scheduled process in the same application or create a new module(a spring boot app) for it. I appreciate your suggestions.
Spring boot scheduling is very handy, go with it if you can. However if you have multiple instances of the spring application, scheduling will be enabled on all of them making it hard to synchronize who calls what.
Now if you are running multiple instances and assuming you have some sort of load balancer in front of them, than create a separate cron job which will call some endpoint and your load balancer will rout the request to one particular instance.
However in the case with multiply instances, probably the cleanest way is to use some sort of messaging like queues. Then all of the application could subscribe to a topic and your cron can just push notifications to it.
as we know the struts interceptor execute and wait will take care of long running process by not getting the request to timeout and destroy it sends wait and at last the desired response i want to implement the same for long running process in spring and hibernate.
I recommend you to use DeferredResult of Spring. It´s a Future implementation, that use the http long poling technique.
http://docs.spring.io/spring-framework/docs/3.2.0.BUILD-SNAPSHOT/api/org/springframework/web/context/request/async/DeferredResult.html
So let´s says that you will make a request, and the server it will return you the deferredResult, and then your request will keep it open until the internal process(Hibernate) finish his task. The timeout is configurable in the constructor.
Here another example http://www.javacodegeeks.com/2013/03/deferredresult-asynchronous-processing-in-spring-mvc.html
In order to keep the session open throughout the lifetime of a request we tie it to the view. This is done either by using Spring's OpenSessionInViewInterceptor or OpenSessionInViewFilter
An open session in view filter will ensure that the Hibernate session is kept open all the way up to the rendering of the view.
Or
You can use a task queue in the backend for a long running process like this.
Work Queues (aka: Task Queues) is to avoid doing a resource-intensive task immediately and having to wait for it to complete. Instead the tasks are scheduled to be done later. Task is encapsulated as a message and sent to a queue. A worker process running in the background will pop the tasks and eventually execute the job. When you run many workers the tasks will be shared between them.
This concept is especially useful in web applications where it's impossible to handle a complex task during a short HTTP request window.
Here's what i need.. I have a UI where a user has the capability to upload a file and extract a report based on the inputted(uploaded) data. Since there is a huge data to be extracted, once the user uploads the data i would like to come out of the servlet control so that user doesn't have to wait in the same page and that the control to be passed on to a java stand alone program there by making it possible for the user to work on something else. So once the control goes on to the java standalone,it would invoke back-end sps and build an extract out of it and place it in a file path on the server.
The user how-over has a capability from UI to check if the extract is ready for them to download.
So the question here is, what is the best practice or possibility in achieving the same? Please let me know your valuable comments.
Thanks!
If you're running in a Java EE environment I would suggest having the servlet dispatch the task to a JMS queue and use a message driven bean to do the (async) processing.
As others suggest, it would be fairly trivial to have the upload servlet redirect the user to some ajax-enabled page that polls the backend for job completion.
If you're not in an EE environment, you could create a standalone (thread pooled) application to consume from the queue and provide signalling eg. through the database (I assume the result goes in a DB anyway). The Spring framework provides very capable and extensive facilities for binding it all together.
But really, there are several free/open source EE containers available, from light weight up to enterprise, so there's no need to build the necessary stuff yourself.
Cheers,
Its very easy.
Have one thread in your servlet class.
Run the thread (Thread will extract the data etc).
After running the thread redirect user to a page where you have auto-refresh or something to show how much extraction is done.(You mentioned that you have a way to find it)
If you can't use message driven beans, you could have your servlet upload the data to a location on the filesystem and record a row in a DB table to say there's a job to be processed.
Then you have your standalone program polling for jobs, processing the data and updating the DB row on completion (including reasons for failure etc.).
Finally, you can poll the status of the job from the UI using an ajax request.
Allows the user to build up a queue of data jobs to be processed while they're doing something else.
I'm very confused about GAE's concepts of Tasks, Task Queues (both push and pull), Cron Jobs and how each of these relate to Frontend vs. Backend instances.
I'm trying to achieve a situation where some HTTP requests can be serviced immediately, whereas some get queued. Queued requests might ultimately end up triggering my own code to execute (once they are consumed) or they might hit one of the GAE service APIs (LogQuery, etc.).
I can't seem to wrap my head around how to design these two scenarios and let alone do the code up. To make things worse I've read literature that suggests there's certain task/queue-related coding you want to do differently depending on whether the code is executing on a Frontend or Backend instance. Thanks in advance for any help here! Bonus points for some concrete examples!
You write the code, Tasks and Cron execute it.
Task is a wraper for a set of properties, the main one is Url that should execute. You code (handler, servlet) should reside on that url. Task sit in the TaskQueues, which have certain default properties on how fast, how many in parallel, etc.. they execute the Tasks. So basically a To-Do list, that sequentially executes tasks with no guarantee when a task will start.
Cron is a service that periodically calls Url that you provided. In a sense its a scheduler.
Your Url (= your handler/servlet) can reside on frontend instance (default) or backend instance (must set special property on Task or in Cron settings). The main difference being that front requests must complete in 10min, while backend requests can take indefinitelly.
What is the purpose of Task Queue Java API? How does it work, and where should it be used?
The homepage looks pretty unambiguous:
With the Task Queue API, applications
can perform work outside of a user
request but initiated by a user
request. If an app needs to execute
some background work, it may use the
Task Queue API to organize that work
into small, discrete units, called
Tasks. The app then inserts these
Tasks into one or more Queues. App
Engine automatically detects new Tasks
and executes them when system
resources permit.
One thing GAE does is keep your request-response cycle very short lived to increase scalability. That is why lot of things like database access and http requests are handled asynchronously.
However there are requests that just can't be fully handled in realtime. This is either because such requests do some long computation (so they can be done in the background) or because they are periodic tasks like cron jobs that you need to schedule and execute repeatedly.
Tasks Queue let do you that.