I have created a cron.xml file and a servlet which describes the job.
Now when i compile and login as an admin, local development dashboard doesn't show Cron Jobs link.
Local development server does not have the Cron Jobs link neither does it execute cron jobs. The actual appengine will show cron jobs and will execute them.
You can manually execute cron jobs on local server by visiting their urls. e.g.
http://localhost:8888/FindReservedBooksTask.
BTW the cron.xml file should be in the war/WEB-INF directory.
The dev appserver doesn't automatically run your cron jobs. You can use your local desktop's cron or scheduled tasks interface to hit the URLs of your jobs with curl or a similar tool.
Here is the link to the GAE doc on this.
Also, make sure you disable all security constraints for your crons in the web.xml. If you don't have them -- you should restrict cron to admin accounts.
This website walks you through the way to use cron jobs inside of Google App Engine.
Also, The Google App Engine provides a service called the Cron Service that helps us do two fundamental things:
Allows your application to schedule these tasks.
Execute these tasks based on their schedule.
Related
I am working on an application which is deployed in pcf and we are using caching mechanism. We want to implement a batch job which will purge the data from cache region.
I want get some suggestions on below:
should i include this batch job in the same application so that it uses the same server to run the batch jobs?
or should i create a new server for running these batch jobs?
Just want to see how the performance of the current application will be impacted, if we run the batch job from the same server. Just want to see the advantages and disadvantages for the same..
TIA
I am new to Flink and kubernetes. I am planning to creating a flink streaming job that streams data from a FileSystem to Kafka.
Have the flink job jar which is working fine(tested locally). Now I am trying to host this job in kubernetes, and would like to use EKS in AWS.
I have read through official flink documentation on how to setup flink cluster.
https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/deployment/kubernetes.html
I tried to set it up locally using minikube and brought up session cluster and submitted the job which is working fine.
My questions:
1)Out of the two options Job cluster and session cluster, since the job is streaming job and should keep monitor the filesystem and when any new files came in it should stream it to destination, can I use job cluster in this case? As per documentation job cluster is something that executes the job and terminates once it is completed, if the job has monitor on a folder does it ever complete?
2)I have a maven project that builds the flink jar, would like to know the ideal way to spin a session/job cluster using this jar in production ? what is the normal CI CD process ? Shall I build a session cluster initially and submit the jobs whenever needed ? or spinning up Job cluster with the jar built ?
First off, the link that you provided is for Flink 1.5. If you are starting fresh, I'd recommend using Flink 1.9 or the upcoming 1.10.
For your questions:
1) A job with file monitor never terminates. It cannot know when no more files arrive, so you have to cancel it manually. Job cluster is fine for that.
2) There is no clear answer to that and it's also not Flink specific. Everyone has a different solution with different drawbacks.
I'd aim for a semi-automatic approach, where everything is automatic but you need to explicitly press a deploy button (and not just a git push). Often times, these CI/CD pipelines deploy on a test cluster first and make a smoke test before allowing a deploy on production.
If you are completely fresh, you could check the AWS codedeploy. However, I made good experiences with Gitlab and AWS runner.
The normal process would be something like:
build
integration/e2e tests on build machine (dockerized)
deploy on test cluster/preprod cluster
run smoke tests
deploy on prod
I have also seen processes that go quickly on prod and invest the time in better monitoring and a fast rollback instead of preprod cluster and smoke tests. That's usually viable for business uncritical processes and how expensive a reprocessing is.
I have a core java application which is using Quartz 2.2.1 with JDBC job store. All the jobs are scheduled in the same.
I am building another Spring based application using Appfuse, maven and Quartz.
I want to reschedule the jobs running in the former application from the spring application.
While doing that I am getting class not found exception as I have not added the job classes in the class path.If I am adding them I am able to update the jobs.
Is there any way to manage the jobs from Spring application without adding the job classes in the class path.
I do not want to update the quartz database using jdbc or hibernate.
Yes, that is a known limitation of the Quartz remote API. It becomes very painful if you have to communicate with or manage multiple Quartz scheduler versions remotely. It becomes even more painful when the management application uses the Quartz API internally (which seems to be your case).
If you take a look at the QuartzDesk project that I founded, you will find out that it very elegantly solves the problem by exposing a JAX-WS SOAP interface through which you can communicate with and manage external Quartz scheduler instances. It hides all Quartz scheduler API complexities and Quartz differences behind a simple Quartz-like API.
The JAX-WS interface is described here and there is also the relevant WSDL file available for download.
We are developing an application which periodically syncs the LDAP servers of different clients with our database. This application needs to be accessed via a web portal. A web user will create, modify or delete scheduled tasks on this application. So, we have developed this application as a web service.
Now, we have to scale this application and also ensure high availability.
The application is an Axis2 based web service running on Tomcat. We have thought of httpd + mod_jk + tomcat combination for load balancing. The problem is that if a request for modification/deletion comes, then it should land on the same tomcat server on which the task was created initially. But, since, the request can come from different web users accessing web portal from different ip addresses, we can not have same session id (sticky session).
Any solutions? Different architecture? Anything.
We have also thought of using Quartz scheduler api. The site says it supports load balancing and clustering. Does anyone has experience of working on such scenario with Quartz?
If you are using Quartz for your scheduling, that can be backed by a database (see JDBCJobStore). Then you could access any Tomcat server and the scheduling would be centralized. I would recommend using a key in the database that you return back to the Axis service, so that the user can reference the same data between calls.
Alternatively it is not difficult to use the database as a job scheduler, then have your tasks run on Tomcat (any location), and put the results into the database. If the results of the job (such as its status) are small, this would work fine.
I searched around looking for my situation but found many threads on making multiple quartz schedulers on different machines run a job once. My situation is the opposite. We have multiple web servers behind a load balancer, all are using quartz and connect to the same database. One of the jobs is to load log files from a third party app into the database. When the job is triggered only one of the web servers picks it up. I am trying to find a solution to have one scheduled job and when it is triggered all the attached web servers will pick it up and start processing the logs on that machine from this third party app.