I have a simple DropWizard service and I'd like a REST API to start a long running processing task - both CPU and I/O bound. REST call will not wait for task completion, notification will happen by polling/long polling/web socket.
For now, I'd prefer if I can do this in Dropwizard and keep everything in single deployable JAR. What are my options?
UPDATE: I am interested in what my options are regarding running long running tasks in Dropwizard, deployed as single jar without external dependencies. Just spawn a new thread? Assuming there are just few such requests it would probably work but there should be better options.
You probably want to use a managed resource:
https://dropwizard.io/en/stable/manual/core.html#managed-objects
to setup a thread pool. Then, your initial request could push a message onto a queue. Your thread pool can pull messages off the queue and process them asynchronously.
You could maybe provide an additional endpoint so that clients can obtain the current state of the asynchronous process.
Related
We have a lot of automated tasks in a activiti process which takes a lot of time to complete. Like REST calls to external system OR JMS message send and receive response.
I can do this using two options :
Service Async task
Send - receive task: Here we can have one task which will call the external program with correlation id as execution id and move to the next step which would be a receive task. Once we get a response from the external system we can signal the process.
Can someone validate what would be the correct approach or any better way to handle this in activiti.
If you have control over the receiver services, you could also use an intermediate message event to pause execution and have the external (long running) service release the process by injecting a message.
Your two approaches each have advantages/disadvantages
Service Async Task - I am assuming you are proposing to keep the "pause" logic inside the service task and simply block on the task until the external service comes back. While this is neater and keeps the logic inside the service task implementation, it doesn't allow for boundary timer events or other BPMN error/timeout handling to move the process on if the service timesout. Certainly you can throw a BPMNError which will bubble up but it makes the actual logic harder to follow.
Send-Receive Tasks - This assumes you have control over the external services as it is very similar to the third option I mentioned using intermediate message events. I tend to prefer this approach as the logic is clear and obvious. However, it does mean you must have the ability for the external service to "send" something back to the receive task to progress the flow.
Either way, both options have their place and much depends on the nature of your external service.
Hope this helps.
as we know the struts interceptor execute and wait will take care of long running process by not getting the request to timeout and destroy it sends wait and at last the desired response i want to implement the same for long running process in spring and hibernate.
I recommend you to use DeferredResult of Spring. It´s a Future implementation, that use the http long poling technique.
http://docs.spring.io/spring-framework/docs/3.2.0.BUILD-SNAPSHOT/api/org/springframework/web/context/request/async/DeferredResult.html
So let´s says that you will make a request, and the server it will return you the deferredResult, and then your request will keep it open until the internal process(Hibernate) finish his task. The timeout is configurable in the constructor.
Here another example http://www.javacodegeeks.com/2013/03/deferredresult-asynchronous-processing-in-spring-mvc.html
In order to keep the session open throughout the lifetime of a request we tie it to the view. This is done either by using Spring's OpenSessionInViewInterceptor or OpenSessionInViewFilter
An open session in view filter will ensure that the Hibernate session is kept open all the way up to the rendering of the view.
Or
You can use a task queue in the backend for a long running process like this.
Work Queues (aka: Task Queues) is to avoid doing a resource-intensive task immediately and having to wait for it to complete. Instead the tasks are scheduled to be done later. Task is encapsulated as a message and sent to a queue. A worker process running in the background will pop the tasks and eventually execute the job. When you run many workers the tasks will be shared between them.
This concept is especially useful in web applications where it's impossible to handle a complex task during a short HTTP request window.
I am looking for suggestions or ideas.
There is an external process (or even a browser) that needs to trigger a long-running process via simple web service call that ideally should run in the same container as that web service. We're using Apache ServiceMix. The web service itself shouldn't stay alive for the duration of the long-running process, besides it may just time-out anyway so we want it to return the response normally pretty much right away.
Originally, I was thinking of using ProcessBuilder() to launch the long-running process as just another app but doing this introduces certain OS dependencies and seems like a less then ideal practice anyway. One of the options we considered is starting another thread from the request and just letting the request complete immediately with a response while the long-running thread would keep on going as long as needed. I fear resource hijacking on the container as well as long-running thread's health when its launcher/parent exits losing any reference to that long-running child.
If anyone has any good ideas for how this can be solved in an elegant way, please let me know.
Thank you very much!
I'm guessing here as you didn't provide the version of your servicemix. Though with Camel which is included with servicemix I'd have two routes the first one providing the web service the second one doing the long running process. The second route should use the seda component. This will give you the async call.
I would like to ask what is the best approach to run a long process from a java servlet. I have a webapp and when the client do a request it runs a servlet. This servlet should get some parameters from the request and then runs a process. This process may take a long time so I need to run it separately. When this process executed finish, it send an email with the results.
Thanks in advance.
Use a thread pool. Each time you receive a request, create a task and submit it to the thread pool. This will ensure too many requests don't bring the server to its knees, because you'e in control of how many concurrent threads you can have, and how many tasks can wait in the thread pool's queue of waiting tasks.
See the javadoc for Executors and ThreadPoolExecutor.
Though this sounds a bit dangerous that invocation of a servlet spawns a process (without proper throttling capabilities in place), you can spawn a process using Runtime.getRuntime().exec(). Much better would be to use ProcessBuilder to prepare the process arguments and spawn it.
Normally that kind of activities is delegated to another type of application module like a message driven bean and that seems to be the cleanest, and standards compliant solution to me. Although most servers won't complain if you create your own threads (which is forbidden by the standard but rarely enforced) the amount of management needed to set up your own job queue and pooled execution environment isn't really worth it in my opinion.
I see two possibilities to do this:
Create a separate thread for each task (thread pool approach). This is possible, but potentially may create a performance problem.
Create a second application. For instance you can save parameters to DB. Second application will monitor this DB with some interval and do something. Instead DB you can use some message queue manager like WebSphere MQ
Second approach have the advantage: if app not able to process the request now by some reason, the app can return to it later
I am using jetty, version 7.0.1 if that matters.
Sometimes I have some quite long running tasks on a server which I would like to cancel/stop if the client disconnects (in case of GET requests, not e.g. POST file uploads). It seems this is not the case, and that tasks continue to run to
completion.
Perhaps I can use ServletRequestListener.requestDestoryed listener to get notification of such tasks but what is recommended
approach for stoping the request thread? What about releasing resources like database connections, file handles or running tasks
(executor service)?
What is the recommended approach in stopping such tasks as soon as possible?
first I would recommend updating to the latest versions of jetty, we have fixed a ton since 7.0 series
second, your best bet to solve this problem is by design using either jetty-continuations to get async servlet support with servlet 2.5 spec (which is jetty7) or update to servlet 3.0 (jetty 8) and not rely on the get methods of the servlet api to block waiting for a response to send. Instead process the request and then spawn a thread or use an executor future to process the actions, then calling back to the request when you have a payload or success message to return. Reason being that while your in the servlet api blocking on the request process you are consuming resources and threads from your servlet thread pool...you'll be able to scale up much cleaner by using continuations or the async servlets of 3.0...
Also you'll be able to design a proper mechanism for managing these threads and things like timeouts and the proper notification mechanism for exceptional conditions, and it will be testable outside of a servlet container that way.
imo at least :)