I have a parent job that triggers many downstream jobs dynamically.
I use python code to generate the list of jobs to be triggered, write it to a properties file, Inject the file using EnvInject plugin and then use the "Parameterized trigger plugin" with the job list variable (comma separated) variable to launch the jobs (If anyone know an easier way of doing this I would love to hear that also!).
It works great except when killing the parent job, the triggered jobs continue to run, and I want them dead also when killing the parent.
Is there a plugin or way to implement this? Maybe a hook that is called when a job is killed?
EDIT:
Sorry for the confusion, I wasn't clear about what I meant with "killing" the job. I mean clicking the red 'x' button in the Jenkins gui, not the Unix signal.
Thanks in advance.
Instead of killing the job, have another job that programmatically terminates all the required jobs. You could reuse the same property file to know which all jobs to be killed. You could use groovy script to terminate jobs.
To catch SIGTERM inside your process you could use the following code (unix specific):
import signal
def kill_children(*args, **kwargs):
# some code that uses the stored list of children procs to kill them
signal.signal(signal.SIGTERM, kill_children)
There are lots of other signals that a process can receive. SIGKILL is the most obvious in your situation. So it would just be a matter of working out what signal was killing the parent and handling it.
Related
I have a general design problem regarding Cucumber-
I'm trying to build some cucumber scenarios around a specific external process that takes some time. Currently, the tests look like this:
Given some setup
When I perform X action
And do the external process
Then validate some stuff
I have a number of these tests, and it would be massively more performant if I could do the external process just once for all these scenarios.
The problem I'm running into is that it doesn't seem like theres any way to communicate between scenarios in cucumber.
My first idea was to have each test running concurrently and have them hit a wait and poll the external process to see if it's running before proceeding, but I have no way of triggering the process once all the tests are waiting since they can't communicate.
My second idea was to persist data between tests. So, each test would just stop at the point the process needs to be run, then somehow gets their CucumberContext to a follow up scenario that validates things after the process. However, I'd have to save this data to the file system and pick it up again, which is a very ugly way to handle it.
Does anyone have advice on either synchronizing steps in cucumber, or creating "continuation" scenarios? Or is there another approach I can take?
You can't communicate data between scenarios, nor should you try to. Each scenario (by design) is its own separate thing, which sets and resets everything.
Instead what you can do is improve the way you execute your external process so instead of doing it each time, you use the results of it being done once, and then re-use that result in future executions of the scenario.
You could change your scenarios to reflect this e.g.
Given I have done x
And the external process has been run for x
Then y should have happened
You should also consider the user experience of waiting for the external process. For new behaviours you could do something like
When I do x
Then I should see I am waiting for the external process
and then later do another scenario
Given I have done x
And the external process has completed
Then I should see y
You can use something like VCR to record the results of executing your external process. (https://rubygems.org/gems/vcr/versions/6.0.0)
Note: VCR is ruby specific, but I am sure you can find a java equivalent.
Now that your external process executes pretty much instantly (a few milliseconds) your no longer have any need to share things between scenarios.
For example if i have a pool of 10 threads and these threads go out and start different processes (perl scripts), is there a way that those threads can "check up" on those scripts to see how they're doing?
Sometimes some of the scripts freeze up and I have no way of knowing. So i've been thinking of a way to have the threads check on the scripts every once in a while so I can be notified when a script is hung up so that i can start figuring out why they are hanging up and fix the problem.
For example if i have a pool of 10 threads and these threads go out and start different processes (perl scripts), is there a way that those threads can "check up" on those scripts to see how they're doing?
The only way that I know of to do this is if the script itself output some sort of status message every X seconds to standard-out or error and the thread that spawned the script was actively reading, waiting for the output. It then could update status information about the process.
If you use the ProcessBuilder and call start() to get a Process, then you can attach a BufferedReader to the process.getOutputStream() to monitor the script. You certainly can also call process.exitValue() to see if the process has finished but that won't tell you if it is hung.
Alternatively would be for the script to somehow call back to the monitoring process via a socket or some other IPC but I think just monitoring the output-stream is the best approach.
Actually, as I posted in a comment, knowing if a program is "hung-up" is an undecidable (can't be solved) problem called The halting problem. So it is not an option to verify on this.
Some alternate solutions would be to check if the script is still running by calling the isAlive() method on the Thread object that is running the script, or, as was told in other answers, to check for some output that the script might be giving, and interpret it. But by verifying output unfortunately you cannot be sure that the program is "hung-up", you can only know it's current state.
EDIT: If you want to wait a particular time, then you can use the Thread.sleep(long millis) call on the parent, and when it wakes up, check who's alive using, again isAlive(). But this doesn't guarantee either that the program will actually finish
I have a .bat file in a Windows machine that starts our program by calling a main class of a Java executable(.Jar)
Now I need to run this every 30 mins.
I gone through several ways of doing it, but unable to decide which is better.
Scheduling through Windows scheduler or Using Java Timer. Which one to choose?
I want only one instance of the process running. If the previous process doesnt complete within 30min, i could wait.
Please let me know what to go for, based on my use case.
Thanks in advance.
You're better off using the Windows Scheduler. If there's a real risk of the process taking too long, you can create a file, or open a socket while the process is running and when another one tries to start up, it can detect that and simply quit. This would make it "miss" the 30m window (i.e. if the first job started at 12 and finished at 12:35, the next job would not start until 1).
But this way you don't have to worry at all about setting up long running processes, starting and stopping the java service, etc. The Windows scheduler just makes everything easier for something like this.
TimerTask is not a scheduling system, it is a library that provides tools for in-app scheduling. Seems that for your use-case you need a the system: you need it to run whether or not your app is running, you need reporting, etc. Windows Scheduler (or cron on unix/linux) is more appropriate for your needs.
I have a Java Swing application that is being used as a cluster application. The problem is that every time the cluster tries to terminate the Java application, it just hangs and Windows displays the "End Now" dialog. The said application is a server type one so it spawns a thread for every attempt to connect to it is made.
I learned that the cluster sends the TERM signal using the program presented in this article. BUT when the console application is used as a cluster application, the cluster can just terminate the process after a few TERM signals.
I also tried the vanilla sample desktop application that's available when making a new project using NetBeans 6.8. It also won't terminate even after receiving the signal.
From the demonstrations done above, I think that it has something to do with Swing or with the threads. Can anyone help me with this? Thank you.
EDIT: It could be killed by using the task manager though I think it sends another signal.
When your Java application receives the TERM signal it will run any registered shut-down hooks before terminating. One possibility is that one of these shut-down hooks is blocking indefinitely or else taking a long time (>30 seconds) to run, causing the Windows "End Now" dialog to be displayed.
One thing you could try is to register a shut-down hook that simply prints to the console and verify that it is indeed being called. However, unfortunately there'll be no way to determine whether other shut-down hooks have run at this point as hooks are run in an arbitrary order.
What's the best/easiest way to run periodic tasks (like a daemon thread) on a tomcat/jetty server? How do I start the thread? Is there a simple mechanism or is this a bad idea at all?
If want to keep everything on java side, give a look to Quartz.
It handles failover and fine grained repartition of jobs, with the same flexibility of cron jobs.
It's okay and effective to stash a java.util.Timer (or better yet ScheduledExecutor) instance in your ServeletContext. Create it in a Servlet's init() call and all your servlets can add TimerTasks to it.
One general purpose way which works for many systems is simply to have a cron job which performs a periodic wget against your app.
I can't answer the tomcat/jetty stuff, but I've done similar things with Python based web apps.
I normally just run a separate app that does the periodic tasks needed. If interop is needed between the website and the app, that communication can happen through some sort of API (using something like XML-RPC/unix sockets/etc) or even just through the database layer, if that's adequate.
Hope that helps.
If you want to use a cron job but don't have administrative access to the development system, you can do a user crontab by executing the command:
crontab -e
It uses vi by default on most systems, but you can change it to the editor of your choice via:
export EDITOR=/usr/local/bin/my_editor
Then, executing the crontab -e command will launch your crontab file in your editor. Upon saving, the changes will be committed back into the system's cron.