What is the purpose of Task Queue in google app engine - java

I am working on project using GWT, Google app engine. I spent whole day for understanding task queue but i did not understand. could you please tell me what is the purpose of Task Queue. And now i am using automatic scaling so server side request processing limit is Only 1 minute so Using task queue shall i process my request more than 1 minute using Task Queue on server side? Any help??
Thanks in advance

In short, task queues are for "work outside of a user request, initiated by a user request" making them good for "background work." Instead of the 1 minute limit for normal requests, Tasks can run for up to 10 minutes. That does mean that you might not have the results of the Task before a typical user request times out (or throws the DeadlineExceededError).

Related

how to implement countdown trigger in server side using java?

Bidding sites like quibids and ebay has a countdown showing how much time left for the auction. I know this can be taken care on FE and should be fairly easy. What I want to know is how to do this on server side? like sending an email to people participate but didn't win and updating database when times up. I've thought about two approaches to do this.
keep the timer on client side and do updates when the first request hit
open a new thread and make it sleep for x amount of time then wake up to do the updates.
Both approaches don't sound right to me and will lead to issues I think. Like user will likely not getting the updates on time, or server will have lots of sleeping beauty waiting.
What I want to know is how to do this on server side? like sending an
email to people participate but didn't win and updating database when
times up.
The best way may vary depending of technology stack of your server side.
You if are running from a Servlet container (e.g.: Tomcat, Jboss...), you
probably want to do something similar to this: Background timer task in JSP/Servlet web application
If you are running a Spring application (e.g.: Spring Boot or Spring MVC), then I recommend #Scheduled or other Task Execution and Scheduling
For advanced scenarios you may want to go with Quartz
Something else, then you should try hooking it up with Java Timer Task
To schedule a task, use a ScheduledExecutorService.

Is it possible to create a GAE cron job on a local GAE server?

This link says that GAE does not support cron jobs on the development server. So what are my other options?
I need to send emails out to users when a deadline on their account is reached.
Use a local cron service, and whrite a script which uses curl or wget to call your cron handler.
Another way is to use enqueued tasks.
At start-up, your web app could query all users and enqueue a task for each one if there is a deadline, with the task's ETA set to that deadline.
When a task fires, it checks to see if the deadline is "now" or earlier. If so, then execute your deadline processing code. In all cases, examine (the next?) deadline for that user. If one exists then enqueue another task....
Of course, your dev web app server must be running continuously for the enqueued tasks to fire; they are all lost on shut-down.
Any help?

Error Page before response processing finishes

I have certain use case where processing can take upto 2 hrs of time. so once user submit the request from browser, my understanding was browser would keep on waiting for response to arrive. But I get error page after some 15-20 mins.
I understand that web request should not so much time consuming, but i stuck with the existing architecture and design.
Can some one suggest some solution for this problem.
I am using IE 9 as browser. Tomcat as server.
What you could do for similar issues is create a separate thread on the server and return a response to the user saying that job has been started and then either
display the result of that job on a specific page (this seems like an acceptable solution, the user will probably not stay in front of his screen for such long task)
or via ajax do some polling to have the status of the job you just triggered.
Most probably the server timeout is about 15 min, therefore you get the error after 15 min. One solution is to increase the server timeout. But increasing to 2 hours would be too long. Another option is to poll the server from the browser to find out the status of the task. You can use ajax call for the purpose.

GAE :Process terminated because the backend took too long to shut down in backends job

My backend job is working on the basis of cron job(every 4 hour).But it is terminated with out processing the data. The server log displays as following :
500 15377121ms 0kb instance=0 AppEngine-Google; (+http://code.google.com/appengine)
E 2012-10-05 01:50:18.044 Process terminated because the backend took too long to shutdown.
How to handle this kind of error in my program
That error is generated when App Engine needs to shut your backend down but the backend fails to exit within 30 seconds. Some reasons why this might be happening are listed here. Depending on the type of error, App Engine may be sending your backend a notification of the impending shutdown, so it's a good idea to register a shutdown handler so you can gather more data about your app's state when this is about to happen.
If you are seeing this regularly there is probably a systematic explanation, such as your job's memory exceeding the maximum for the backend's class.
Dealing with the same issue. Looked at the causes listed in the official docs. Memory consumption seems ok from statistics. Datastore contention issues are also handled in my code. Timeouts too. Changing the task mechanism to work in recoverable chunks seems the only way out.
After chasing this error for sometime now, it seems AppEngine development paradigm revolves around url handlers with limitations on time, memory etc. This is applicable to long running tasks too. I redid my long term task to be executed small tasks. Task queues triggered smaller tasks which inturn run, before finish queue the next task. Never failed even once!
The advantage is that taskqueuus have better failsafe/handover than just a huge cron job. One task failing does not mean the rest of the huge task list fail.

Triggering an Asynchronous process in Java

I have a web application that takes in user requests and puts them into a MYSQL database. Now a typical user request needs to be serviced by following a workflow that would take significant time to complete. To address this i have an asynchronous processor that keeps listening to the MYSQL table.
I have noticed that polling the MYSQL table on an infinite loop results in a spike in CPU usage on the box my application is deployed to that often renders the box unusable.
I know that making the asynchronous process sleep for 'some' time whenever there aren't any active requests in the MYSQL database is an option but i would like to keep that as a last resort.
Making this process synchronous is not an option because of the time the workflow involved in servicing a single request takes and also because there is a need to decouple the processing from the front end to allow the back end to evolve.
I would like to know if there is any smart way to trigger off the asynchronous process so that i can avoid the CPU usage spike and still get optimum response time from the asynchronous processor.
Any advices would be appreciated.
Thanks
p1ng
An option would be to store the request in the database AND send some kind of event in you system (eg. JMS message, or by using java.util.concurrent constructs). The process that reads and executes the command can then be awaken by this signal, go fetch the data in the database and process it as usual.
This way, you wouldn't waste CPU cycles polling not-yet-available data, and you would be more reactive due to the absence of polling delay.
You can make your asynchronous process read from a TCP socket or something similar. The asynchronous process should just wait blocking on i/o. Then from your primary process you can send a message over to the asynchronous process once it has updated the table. It may be possible to send the message from a trigger in database too.
I would generally not recommend polling based approach to check the table. What happens when you have to poll for different events at different schedules? If you envision need for multiple events in the future, I would suggest looking into message queues for asynchronous tasks.
But if you have to go with polling based approach for now - I don't fully understand your reasoning against letting the asynchronous process sleep for some time? Why would you want your process to consume all the CPU resources doing nothing but running in an infinite loop? Why not have your async process run at specific intervals? You can make this polling interval configurable.
You can use the FutureTask Java API to do that. See that blog entry.
Or perhaps just new Thread(YourRunnable).start(), and make some state variable in YourRunnable to know if your task is finished or not.

Categories