My Grails 1.3.7 app needs to process big XML files, so I have a controller to which I upload the big file, and then I give the path of this file on the server to a background thread that does the processing so that I can return right away from the controller action.
For now, I am using the Grails Executor plugin and it works fine. I have an Upload domain object that is updated as the processing progresses (current status, number of processed elements, etc.). But now I have 2 more requirements:
when the application crashes or the server is shutdown, I would like to intercept that and update my Upload domain to say that the process was interrupted
I want the user to be able to interrupt the processing himself when clicking a link and possibly resume it from controller actions
Is there a way that I can persist a reference to my background task and intercept any interruption with java.util.concurrent framework (which is used by Executor plugin)?
And if I can't do it with util.concurrent, is it possible with other plugins/frameworks? I've had a look at Quartz for example, but I don't see how to do it.
I hate to answer without fully testing it, but the grails-executor plugin docs state that the callAsync method returns a java.util.concurrent.Future object.
This object can be used to do two things:
Determine if a process has completed or been canceled.
Cancel a running process (even interrupting if necessary).
In theory, you should be able to save this Future in your user's session somewhere. Then you could retrieve it later and use it to check the status and/or cancel the process as necessary.
Something like:
session.backgroundProcess = callAsync{...}
// later
def bgProc = session.backgroundProcess
if(bgProc && !(bgProc.done || bgProc.cancelled)) {
// process is still running
}
That's just the idea, it's not tested. I also don't know if there are issues with this leading to memory leaks. You'd need to make sure you detached the Future once the process is completed.
Related
This Q is asking for resources/direction rather than a specific answer. we're looking to understand the Vaadin architecture on the following problem.
Our application is event-driven, event here being a server-side event.
When our application hears from another, independent application on the server, it asks the front-end (human) user
to fill out a form, waits till the user does, and proceeds based on what the user said.
Our backend process, say processB is calling a method on a Vaadin layout to do the deeds. that method, say methodA() is
supposed to
push the form to front-end
wait till the user enters data and hits the "OK" button
process&return what the user entered.
(1) is working fine. however, the method is returning before the front-end form is processed. This and some other stuff we observed here suggest that Vaadin is running
two different threads for methodA(). So, when we invoke
ourUI.methodA();
2 different threads are initiated and running to execute it.
Can this be true?? What is the Vaadin logic on this?
it isn't a thread from a previous run of the application still remained on the server-- i checked that carefully
What's the best way to make a process wait for a Vaadin event? is there a way to make an independent thread, i.e., a thread that is outside the control of Vaadin UI, wait for a Vaadin event?
we can make processB join methodA() so that it'll wait until methodA() returns. but I don't think this is the best way. and i don't know how to get hold of a Vaadin thread to join() it. furthermore, those (seemingly) 2 methodA() threads are spoiling it.
we can keep a token for what methodA() returns and make processB keep an eye on it, wait till value is written there. still-- there must be a better way.
We'll merge processB into Vaadin UI (early days yet in development) so that it'll keep running until the UI is terminated. It no longer will be outside the ourUI. How does this simplify things?
Note: we've got only one UI for one session/user-- not allowing multiple tabs by specs, and one backend process on one UI/user
We're using Vaadin 7.7 on Java 1.7 and Tomcat 8
//-------------------------
EDIT:
Our backend process, processB is a Runnable itself and controls the whole logic. it has to wait for the front-end thread return and deliver the value. what i'm looking for is how to trigger this-- getting the value when front-end event returns. processB won't proceed until it gets this value from the front-end.
I'm developing an App Engine application where an http request spawn a undefined number of tasks (can be 10, can be 100, can be 1000).
Those tasks will run on PUSH queue.
When all the task has finished, I need a conclusive action which will get the status of each task and creates a final report (a summary of all executed operations). For this example let's say it is an email delivery.
An additional info is that the task are grouped by a common value (let's call group_id). When all the tasks of the same group_id have finished, the event can be executed for this particular group. This specification I made is because if other tasks are still running but there are of other group_id, the event for this group can be executed as well.
I'm wondering which is the best approach to implement something like this.
Of course there can be a lot of "homemade" solution where (for example) I save the status of each task inside a Datastore entity and retrieve each status or else each task can "question" its brothers and check if it is the last one.
My question is if there is a more high-level solution to implement this behaviour.
With a quick search I found the pipeline API [1] which seems to be promising, but I'm not sure to understand if it the correct solution for the behaviour I need.
On stackoverflow this library is very well acclamed, so maybe is the right way to do it.
Here some of the thread I read
How can I tell if a set of app engine tasks have all completed?
Writing to an appengine blob asynchronously and finalizing it when all tasks complete
Is there a way to know when a set of app engine task queue tasks have completed?
If I understand correctly the implementation for this Pipeline API, when a Job starts it provides a jobId to test if it has finished or is still running.
If so, this requires an external poll on the job status, which is not my case. My particular need is when the last finished, the final event is automatically triggered
[1] https://github.com/GoogleCloudPlatform/appengine-pipelines
You can use a single Datastore entity to store group_id, the number of tasks started and the number of tasks completed. Upon completion each task updates this entity, and if the number of tasks completed equals the number of tasks started, creates the "final" task to generate a report.
My project is not using Akka as of now, it runs as a single process and does too many things.
As a first inital step, we are trying to put a scaffolding in place so that we get Fault Tolerance. For example,
ProjectSupervisor
|
ExistingProcessActor
Where ExistingProcessActor would run as a forever long running job anf if it gets killed because of some Exception, ProjectSupervisor will restart it
I wasn't too sure about this, so I asked on user-group, and received an interesting advice as
[Me] def run: Unit = LogReaderDisruptor.main(Array())
is a method that is supposed to run forever, plus it required some setup (that is available on client's machine or test environment)
[Advice here] By that do you
mean that the main() is never returning? If so, then you're blocking
the Actor and wasting an entire thread. Spawn this off on a dedicated
dispatcher instead (see dispatchers and futures docs).
I went through the documentation but did not understood what I need to do.
Does that mean following?
ExistingProcessActor will start a new future by giving a custom dispatcher of type PinnedDispatcher?
If so, then in case of any failure how would ProjectSupervisor be notified? He is monitoring ExistingProcessActor and not future (future is not Actor)
I am confused, kindly guide me
You don't need to use a Future in order to have the work done in a dedicated dispatcher. This can be done by spawning an actor and assigning that actor to the dedicated dispatcher. A simple example from the akka docs:
val myActor =
context.actorOf(Props[MyActor].withDispatcher("my-dispatcher"), "myactor1")
http://doc.akka.io/docs/akka/snapshot/scala/dispatchers.html
You could do this within your project supervisor when creating the ExistingProcessActor, putting that actor on a pinned dispatcher, and then your supervision strategy will continue to work as you want.
in a Spring MVC Controller I would like to start a thread that continues work while my controller sends the response. Will this work with spring-mvc ?
Best Reagrds,
Heinrich
Yes, You can start new Thread in Controller. But better way of doing asynchronous job is to use spring-scheduling support. You can leverage Quartz framework. That will manage your job.
This link will give you how to integrate this in your application.
Yes, it'll work. On one web app I worked on, I needed to send a notification email depending on the user's action. I added a post-commit interceptor on the service, and had that fire off the email in a separate thread. (In my case that happened to be cleaner than putting the code in the controller, because I only wanted it to happen if the transaction committed.)
You do need to make sure the thread actually stops running at some point, either by setting daemon to true (if it's ok that stopping the server kills the thread without notice) or making sure the code in its run method will always terminate at some point.
You are better off using a threadpool than creating new threads, so you don't risk resource exhaustion (threads stalling out are usually not independent events, if a thread hangs the next one probably will too, so you need a way to cut your losses). Methods annotated with #Async will be executed using an executor that you can configure as shown in the Spring documentation.
As the others mentioned, it's will work. ExecutorService can do the job. Here you can see I used it for starting a video stream that sits on a separate endpoint.
#PostMapping("/capture")
public ResponseEntity capture() {
// some code omitted
ExecutorService service = Executors.newCachedThreadPool();
service.submit(() -> startStreaming(deviceConfig));
return return ResponseEntity.ok()
.body(stream);
}
Can someone tell me the TRUE difference?
My rule of thumb is that an AsyncTask is for when I want to do something tied to single Activity and a Service is for when I want to do something that will carry on after the Activity which started it is in the background.
So if I want to do a small bit of background processing in the Activity without tying up the UI I'll use an AsyncTask. I'll then use the default Handler from that Activity to pass messages back to ensure updates happen on the main thread. Processing the updates on the main thread has two benefits: UI updates happen correctly and you don't have to worry so much about synchronisation problems.
If for example, I wanted to do a download which might take a while I'd use a Service. So if I went to another Activity in my application or another application entirely my Service could keep running and keep downloading the file so it would be ready when I returned to my application. In this case I'd probably use a Status Bar Notification once the download was complete, so the user could choose to return to my application whenever was convenient for them.
What you'll find if you use an AsyncTask for a long-running process it may continue after you've navigated away from the Activity but:
If the Activity is in the background when your processing is complete you may have problems when you try to update the UI with the results etc.
A background Activity is far more likely to be killed by Android when it needs memory than a Service.
Use Service when you've got something that has to be running in the background for extended periods of time. It's not bound to any activity. The canonical example is a music player.
AsyncTask is great when some stuff has to be done in background while in the current activity. E.g. downloading, searching for text inside a file, etc.
Personally I use Handlers only to post changes to the UI thread. E.g. you do some computations in a background thread and post the result via handler.
The bottom line: in most cases, AsyncTask is what you need.
To complement the other answers here regarding the distinction between service and AsyncTask, it is also worth noting[0]:
A Service is not a separate process. The Service object itself does not imply it is running in its own process; unless otherwise specified, it runs in the same process as the application it is part of.
A Service is not a thread. It is not a means itself to do work off of the main thread (to avoid Application Not Responding errors).
Services tend to be things that describes a significant part of your application - rather than an AsyncTask which is typically contributes to an Activity and/or improves UI responsiveness. As well as improving code clarity Services can also be shared with other applications, providing clear interfaces between your app and the outside world.
Rather than a book I would say the developer guide has lots of good answers.
[0] Source: http://developer.android.com/reference/android/app/Service.html#WhatIsAService
AsyncTask: When I wish to do something without hanging the UI & reflect the changes in the UI.
E.g.: Downloading something on Button Click, remaining in the same activity & showing progress bar/seekbar to update the percentage downloaded. If the Activity enters the background, there are chances of conflict.
Service: When I wish to do something in the background that doesn’t need to update the UI, use a Service. It doesn’t care whether the Application is in the foreground or background.
E.g.: When any app downloaded from Android Market shows notification in the Status Bar & the UI returns to the previous page & lets you do other things.
Service
A Service is an application component that can perform long-running operations in the background and does not provide a user interface. Another application component can start a service and it will continue to run in the background even if the user switches to another application. Additionally, a component can bind to a service to interact with.
When to use?
Task with no UI, but shouldn’t be too long. Use threads within service for long tasks.
Long task in general.
Trigger: Call to method onStartService()
Triggered from: Any Thread
Runs on: Main thread of its hosting process. The service does not create its own thread and does not run in a separate process (unless you specify otherwise)
Limitations / Drawbacks: May block main thread
AsyncTask
AsyncTask enables the proper and easy use of the UI thread. This class allows performing background operations and publishing results on the UI thread without having to manipulate threads and/or handlers. An asynchronous task is defined by a computation that runs on a background thread and whose result is published on the UI thread.
When to use?
Small task having to communicate with main thread
For tasks in parallel use multiple instances OR Executor
Disk-bound tasks that might take more than a few milliseconds
Trigger: Call to method execute()
Triggered from: Main Thread
Runs on: Worker thread. However, Main thread methods may be invoked in between to publish progress.
Limitations / Drawbacks:
One instance can only be executed once (hence cannot run in a loop)
Must be created and executed from the Main thread
Ref Link